Design WhatsApp

Mobile phones cannot be reached by inbound connections, so each device opens one long-lived TCP/TLS connection (WhatsApp historically used a compact XMPP-derived binary protocol; WebSocket or MQTT also work) to a gateway chosen by the load balancer. Gateways are only connection holders: they authenticate, keep heartbeats, and forward frames. Each connect writes device_id → gateway_id into a session registry with a TTL refreshed by heartbeats, so a crashed gateway's entries expire. The router, given a recipient device, reads the registry and sends the frame to that gateway over an internal RPC or message bus. A gateway can hold hundreds of thousands to about a million mostly idle connections; WhatsApp famously ran around 2M connections per Erlang server. Heartbeat interval is a battery-vs-staleness trade-off: send small pings every few minutes, and adapt to the network (NAT timeouts on Indian mobile carriers can be short). Because connections drop constantly on mobile, the design must assume any delivery can fail mid-way, which is why delivery is ack-based from the inbox rather than fire-and-forget.

The promise to the user is: once you see one tick, the message will arrive exactly once, in order. 'Exactly once' over a network is really at-least-once delivery plus idempotent handling. The sender attaches a client_msg_id (UUID); if its send times out and it retries, the router finds the existing record and returns the same ack instead of writing twice. The server persists before acking, so a crash after the ack cannot lose the message. Towards the recipient, the router may deliver a message twice (for example if the device's ack was lost); the device dedups by msg_id. Ordering is per recipient device inbox: server_seq is monotonic per inbox (assigned by the partition owner or using a per-inbox counter), and the device applies messages in seq order and asks for gaps. Global ordering across conversations is not needed. Within a conversation, two people typing at once can produce interleavings that differ slightly between devices; using the sender's timestamp plus server_seq as a tie-break is a standard, acceptable answer. Deletion after ack keeps server storage tiny, which is also a privacy feature.

With E2E encryption the server is a blind relay. Each device has a long-term identity key pair, a signed pre-key, and a batch of one-time pre-keys, whose public parts are uploaded to the key service. To start a session, the sender fetches the recipient device's pre-key bundle and runs X3DH to derive a shared secret without the recipient being online. Each message then uses the Double Ratchet, which gives forward secrecy (old keys cannot be derived from new ones) and post-compromise recovery. For groups, each member creates a sender key and distributes it to every other member device via pairwise sessions; group messages are then encrypted once, which makes server fan-out cheap. Consequences for system design: the server cannot do search, spam filtering on content or server-side message history; multi-device means encrypting per device, so a user with 5 devices costs 5 ciphertexts; adding a device requires a key-change notice ('security code changed'); and backups must be encrypted separately (by a user key or password-protected backup keys). Media is encrypted on the device with a random key; the blob goes to storage and the key and hash travel inside the encrypted message.

Presence looks small but is a fan-out trap: if every status change were pushed to every contact, a user with 500 contacts would create 500 events each time they open the app, and with hundreds of millions of users it explodes. Use pull-on-demand plus subscriptions limited to open chats. When you open a chat with someone, your device subscribes to their presence; the presence service keeps online state from gateway heartbeats in a sharded in-memory store with a TTL and notifies only current subscribers. Last-seen timestamp is written on disconnect and read lazily. Typing indicators are ephemeral frames routed directly to the other party's gateway with no persistence; if lost, nobody cares. Privacy settings (hide last seen from everyone or non-contacts) are checked in the presence service before returning anything. Showing that presence and typing are best-effort and transient while messages are durable is a key signal of judgement.

WhatsApp keeps messages on the server only until delivered; history lives on devices plus encrypted backups. That keeps storage small and supports E2E privacy, but new devices cannot simply download history from the server. Slack and Telegram cloud chats choose server history for search and multi-device sync at the cost of the server seeing content.

Writing one inbox entry per member device makes reads trivial and works because groups are capped at ~1,024. For huge broadcast channels you would switch to fan-out on read: store once per channel and let devices pull.

A compact binary protocol saves bytes and battery on 2G/3G, which matters for Indian users with low-end phones and expensive data. WebSocket with protobuf is a good modern equivalent; JSON is fine for an MVP but wasteful at this scale.

Inbox traffic is append-then-delete at millions of writes per second, partitioned by device. An LSM-based wide-column store handles this well; the cost is tombstones from deletions, which needs tuned compaction and short gc_grace.

Messages per second jump several-fold within minutes, mostly greetings with media. Pre-scale gateways and routers, let queues absorb the burst, prioritise text over media downloads, and shed non-critical work like presence updates and typing indicators first.

One forward into many 1,000-member groups creates huge fan-out. Cap group size, rate-limit forwards (WhatsApp limits forwarding to 5 chats, and 'highly forwarded' messages to 1 chat at a time), and deduplicate media blobs by hash so the same video is stored once.

Billions of deletes per day create tombstones that slow reads in LSM stores. Use time-bucketed partitions or TTL-based expiry, and compaction strategies suited to queue-like data.

Its devices reconnect elsewhere within seconds and resync from their last acked server_seq. Registry entries for the dead gateway expire via TTL, and any in-flight messages remain in the inbox because they were persisted before delivery.

The sender times out and retries with the same client_msg_id; the router finds the existing record and acks. No duplicate.

Users are homed to a region, with inbox data replicated to a second region. On failover, devices reconnect to the other region; a small window of unreplicated messages may be retried from senders' unacked outboxes, which is why clients keep unacked messages until they get one tick.