Design Instagram

Never route large uploads through API servers: they would hold threads and bandwidth for slow mobile clients. The app gets a pre-signed URL and uploads straight to object storage, using resumable or chunked upload so a dropped 4G connection resumes rather than restarts. The app also compresses before upload, which on Indian mobile data saves users money and cuts our ingest. Processing is async and event-driven: an upload event goes to Kafka, and a worker pool creates renditions. Images get 4-5 sizes in modern formats (WebP or AVIF, with JPEG fallback), and a tiny blurhash so the feed shows a coloured placeholder instantly. Videos are transcoded into an adaptive ladder (for example 240p to 1080p) packaged for HLS/DASH, which is CPU heavy and runs on autoscaled or GPU instances. Each step is idempotent and keyed by media_id so retries are safe. Content-safety models (nudity, violence) run in the same pipeline; a post is published only after the required steps pass. Total time target: a few seconds for photos, under a minute for short videos.

Media is more than 95% of bytes, so the CDN is the main serving system. URLs are immutable (contain media_id and size), so they can be cached for a year with no purge needed. The client asks for the size it needs based on screen width and network quality: a 320 px image on 2G, 1080 px on Wi-Fi. This alone can cut bytes by 3-5x on low-end phones. The feed API returns image dimensions and blurhash so layout does not jump. The app prefetches the next few posts' images while the user scrolls and the first second of videos, but backs off on metered or slow connections. Videos use adaptive bitrate so playback starts with a low rendition and climbs. For private accounts, CDN URLs are signed with short expiry so a leaked URL stops working. Origin protection uses a shield tier so a viral post does not cause a miss storm on storage.

Stories behave differently from posts: they live 24 hours, are viewed in a tray at the top of the app, and almost every active user checks them several times a day. Fan-out on write is wasteful because each story expires quickly and is posted far more often than feed posts. So the tray is built at read time: for each followee, check a tiny 'active story' marker (author_id → latest story timestamp) in Redis. With ~200-500 followees this is one batched multi-get. The story contents are stored per author in Cassandra with a 24-hour TTL, so expiry needs no cleanup job. Ordering the tray uses an affinity score (how much you interact with that person), with unseen stories first. The viewers list is a write per view; for big creators this is millions of rows, so it is written to Cassandra with the story as partition key and counted with an approximate counter. Highlights (stories kept on profile) copy the reference to a permanent table so the TTL does not delete them.

A celebrity post can get a million likes in minutes, and every like increments the same counter: a classic hot key. Writing each like as an UPDATE on one database row would serialise on a row lock. Instead separate two things. Membership ('did user U like post P') goes to a likes table keyed by (post_id, user_id), which makes likes idempotent and spreads writes over Cassandra partitions (split hot posts into sub-partitions if needed). The count is approximate in real time: increments are buffered in memory or in sharded Redis counters (post:123:c0 … c15) and summed every few seconds; the total is flushed to the posts store periodically. Users accept that the displayed count lags slightly. Like events also flow to Kafka for notifications ('X liked your post', batched) and for ranking features.

Client-side compression and resizing save upload bandwidth and server cost, but quality varies across phones and you must still re-encode on the server for consistency and safety. Do both: light client compression, authoritative server renditions.

Sharded Postgres works well for posts, users and follows (relational, transactional, well understood); Cassandra suits very high write rates with simple access patterns and TTL (likes, story views, stories). Using both is reasonable when each is matched to an access pattern.

Pre-generating fixed sizes costs storage but makes serving trivial. On-the-fly resizing at the edge saves storage and supports any size, but adds latency on first request and CPU at the edge. Pre-generate the common sizes; resize on the fly for rare ones.

New Year's Eve and Diwali night bring 5-10x story uploads. Prioritise photos and stories (short-lived, users expect them now) over long videos, autoscale workers on queue depth, and publish a low rendition first and add higher ones later.

Petabytes per day is expensive. Move older, rarely viewed media to infrequent-access tiers, delete intermediate renditions that can be regenerated, and deduplicate re-shared media by content hash.

The Kafka message is not committed, so another worker retries; outputs are written to deterministic keys (media_id + size) so retries overwrite safely. After N failures the post moves to a dead-letter queue and the user is told to retry.

Multi-CDN with DNS steering moves traffic to another provider; the app retries image loads with a fallback host. Object storage is replicated across zones.

The post is in the author's profile and visible via pull; followers see it in feed once fan-out catches up. No data loss because Kafka retains the event.