Design a content delivery network

Two ways to send a user to a nearby PoP. GeoDNS: the CDN's authoritative DNS looks at the resolver's IP (or the EDNS Client Subnet hint) and returns the IP of the best PoP, using a map built from latency measurements, PoP load and health. It allows fine control (shift 10% of Delhi traffic to Mumbai) but depends on DNS TTLs (typically 30-60 s) and on the resolver being near the user, which fails when users use 8.8.8.8 without ECS. Anycast: every PoP announces the same IP prefix via BGP, and the internet routes each user to the 'nearest' PoP in BGP terms. Failover is fast (withdraw the route) and it absorbs DDoS naturally because attack traffic spreads across PoPs, but BGP path is not always lowest latency and load steering is coarse. Many CDNs combine them: anycast for DNS and DDoS absorption, DNS-based steering for HTTP traffic. Inside the PoP, an L4 load balancer (ECMP + consistent hashing, like Google Maglev or Facebook Katran) spreads connections without breaking them when servers are added.

With 200 PoPs, a flat design means each object can miss 200 times at the origin. A two-tier hierarchy fixes this: edges miss to a few regional shield caches, and only shields talk to the origin, so a new object causes about one origin fetch per region. Inside a PoP, consistent hashing on the cache key routes each object to one server, so the PoP stores one copy, not twenty, and hit ratio rises. Request coalescing (collapsed forwarding) is essential: if 10,000 users ask for a new video segment in the same second, only one request goes upstream and the others wait for that response. The cache key is usually scheme + host + path + a whitelist of query params; including the full query string or Cookie header destroys hit ratio. Vary headers (like Accept-Encoding) must be normalised to a small set. Storage uses RAM for the hottest objects and NVMe for the long tail, with admission policies (do not cache an object until its second request) to stop one-hit wonders evicting useful content. Stale-while-revalidate and stale-if-error let the edge serve slightly old content while refreshing or while the origin is down.

Purge has to reach thousands of servers fast. The control plane writes the request durably, then publishes it on a global pub/sub fabric (Kafka per region, or a gossip/fan-out tree). Each server applies it and acks. Purge by URL is a direct key delete. Purge by prefix or tag cannot scan 100 TB; instead servers keep a tag index (tag → keys) or store a per-tag/per-distribution generation number and treat older objects as invalid when read. Soft purge (mark stale and revalidate with If-None-Match) is safer than hard delete because the origin is not stampeded. The best practice for customers is versioned URLs (app.3f9a1c.js) with long TTLs, so most deploys need no purge at all. Config changes (new cache rule, new certificate) use the same channel with a monotonically increasing version: servers apply versions in order and the control plane can do staged rollouts, first one canary PoP, then 10%, then all, because a bad config pushed globally is the classic way CDNs cause worldwide outages.

Every customer hostname needs a valid certificate at every PoP. The edge picks the certificate by SNI during the handshake, from a local store synced from a central vault. Automated issuance (ACME / Let's Encrypt) with renewal ~30 days before expiry avoids the most common outage: expired certificates. TLS 1.3 and session resumption cut handshake round trips, which matters a lot on high-latency Indian mobile networks where one RTT can be 80-150 ms. HTTP/2 and HTTP/3 (QUIC) at the edge reduce head-of-line blocking on lossy networks. The edge keeps warm, long-lived connections to shields and origins, so a user in Patna pays the handshake cost only to the nearby PoP, not to a far-away origin. For banks and others that do not want to share private keys, a keyless model lets the edge call the customer's key server only for the signing step.

Pull (fetch on first miss) needs no customer work and caches only what is popular; first requests are slower. Push (customer uploads to the CDN) suits large, predictable files like game patches or a big OTT release, but wastes storage on unpopular objects. Most general CDNs are pull with optional prefetch.

Many small PoPs cut latency (more cities, including Tier-2 India) but each has a smaller cache and lower hit ratio. Fewer, larger PoPs have better hit ratios and are cheaper to run. Common answer: big PoPs in metros plus small embedded caches inside ISPs (like Netflix Open Connect) for heavy video.

Long TTLs maximise hit ratio and origin offload but make updates depend on purge. Short TTLs keep content fresh but raise origin load. Use long TTL + versioned URLs for assets, short TTL + stale-while-revalidate for HTML and APIs.

A cricket highlight goes viral and 1M users request it within a minute. Without coalescing, every edge server misses at once and the origin melts. Fix: request coalescing at edge and shield, the shield tier, and pre-warming objects the customer knows will be hot.

Consistent hashing puts one hot object on one server, which may not have enough NIC capacity. Detect hot keys and replicate them to several servers in the PoP (or serve from RAM on all servers), trading a little storage for spread load.

Terabytes of logs per hour cannot be shipped raw in real time. Aggregate counters at the edge every 10-60 s for dashboards and billing, and batch-compress raw logs to S3 for customers who need them.

Health checks from many vantage points detect it within seconds. Anycast routes are withdrawn and GeoDNS stops returning that PoP; users move to the next-nearest PoP. Neighbour PoPs need headroom (~30-40%) to absorb the shift.

Serve stale content (stale-if-error) for cached objects, return a cached error page with a short TTL for others, and apply circuit breaking toward the origin so retries do not finish it off when it recovers.

Staged rollout with automatic health gates (error rate, CPU) stops a bad config at the canary PoP. Every server keeps the last known good config and can roll back locally without the control plane.