Design an email service (Gmail)

SMTP gives a clear ownership rule: until our MX replies 250 to the DATA command, the sending server owns the message and will retry. So the MX must persist the message durably (replicated queue or a write to the blob store plus a log) before it says 250. If the MX crashes after persisting but before replying, the sender retries and we receive the same message twice. To make delivery idempotent, the delivery service computes a key from (recipient, Message-ID header, hash of body) and does a conditional insert; duplicates become no-ops. Spam work is staged by cost: IP reputation and rate limits during the SMTP handshake (rejects most spam with almost no CPU), then SPF/DKIM/DMARC verification, then content and ML scoring after acceptance. DMARC is what stops someone spoofing 'From: hdfcbank.com': if the bank publishes p=reject and the message fails SPF and DKIM alignment, we reject or quarantine it. Per-user signals (this user marked similar mail as spam) adjust the final verdict. Everything after acceptance is asynchronous and retried from the durable queue, so a slow spam model adds latency but never loses mail.

Almost every query is scoped to one mailbox, so the primary key starts with user_id. Messages, a per-label index and a per-user history log live in a wide-column store (Bigtable at Google; Cassandra or HBase elsewhere). Listing the inbox is a range scan on (user_id, INBOX) ordered by date desc: one partition, a few KB, fast. Changing a label updates the message row, the old and new label index rows and appends a history row; Bigtable gives single-row atomicity, so either keep these in one row family or use a small per-user transaction log so partial updates are repaired. Bodies and attachments are large and immutable, so they go to object storage. Keying blobs by SHA-256 of the content means a 5 MB PDF sent to a 2,000-person company list is stored once. Use reference counts and a delayed garbage collector (days, not seconds) so a race between a new reference and a delete never destroys live data. Per-user encryption conflicts with global dedup; a common compromise is encrypting blobs with keys managed by the storage layer and limiting dedup to within a tenant or organisation.

A global Elasticsearch index over a billion mailboxes would force every query to filter by user across huge posting lists and would make security bugs (leaking another user's mail) a real risk. Gmail-style search instead treats each mailbox as its own small index. Index servers are sharded by user_id; one shard holds the index segments for many users. A query goes to exactly one shard, so p95 stays low even with a million messages. Indexing is asynchronous: indexers consume MessageAdded and label-change events from Kafka and append to an in-memory segment that is flushed and merged periodically, so new mail is searchable within seconds. Labels and flags (is:unread, label:work) are small and change often, so they are better served from the metadata store and intersected with text results, rather than reindexing on every label change. Inactive users' indexes can be evicted to cheaper storage and rebuilt lazily on first search. For operators like from:, has:attachment and older_than:, keep structured fields in the index alongside text terms.

Sending is where large providers earn trust. Outbound workers sign every message with DKIM and publish SPF and DMARC records for our own domain, so receivers can verify us. Queues are partitioned by recipient domain: if a small company's mail server is down, its messages sit in a retry queue with exponential backoff (for example 1 min, 5 min, 30 min, then hourly up to ~5 days) without blocking delivery to Yahoo or Outlook. Workers respect each receiver's connection limits and throttling responses (421/451). Reputation is per IP and per domain, so the service uses separate IP pools: established users, new or suspicious accounts, and system mail. If a compromised account starts sending 50,000 messages an hour, per-account rate limits and anomaly detection stop it before our shared IPs are blocklisted, which would hurt every user. Bounces (5xx) are turned into a delivery-failure message in the sender's inbox. Internal mail never touches SMTP: the delivery service writes directly into the recipient's mailbox.

Mailboxes are a natural fit for a wide-column store: huge total size, simple access by (user, label, time), high write volume, no cross-user joins. Sharded Postgres also works (shard by user_id) and gives transactions for label changes, but resharding exabytes is painful. Use a strongly consistent store (Spanner, Postgres) only for small, critical data like quotas, account settings and filters.

Polling every 30 s from 500M clients is ~17M req/s of mostly empty answers. Instead the server pushes a tiny 'mailbox changed, history id N' signal over WebSocket (web) or FCM/APNs (mobile), and the client pulls the actual changes. The payload stays small, the client always uses the same sync path, and a missed push is fixed on the next app open.

Dedup saves a lot of storage for attachments that go to many people, but adds reference counting, slower deletes and tension with per-user encryption keys. A practical choice: dedup attachments above ~100 KB within a domain or tenant; store small bodies inline or per-message.

References/In-Reply-To headers are accurate when clients set them. Subject-only threading merges unrelated mails titled 'Hi'. Use headers first, subject plus participants as a fallback, and never merge across a long time gap.

A user with 5M messages or a shared support inbox receiving 100 mails/s creates a hot, oversized partition. Fix: split the user's data into time-bucketed sub-partitions (user_id, month), cap rows per partition, and rate-limit delivery per mailbox with a buffer queue.

A botnet campaign can multiply inbound volume 5-10x in minutes. Reject as early and as cheaply as possible (IP reputation, connection limits during SMTP), autoscale the stateless MX tier, and let the ML scoring tier fall back to a cheaper model when overloaded instead of building an unbounded queue.

Changing the analyzer or tokenizer (for example adding Hindi or Tamil stemming) means re-indexing exabytes. Do it lazily per user (rebuild on next search, background for active users first) and keep both index versions during the migration.

MX servers keep accepting mail into their durable queue as long as it has space, and the backlog drains when storage recovers. If the queue fills, MX replies 4xx (temporary failure), and senders retry for days per the SMTP standard. Mail is delayed, not lost.

Do not block delivery forever. After a timeout, fall back to rule-based and reputation scoring, deliver, and re-score asynchronously; a later 'move to spam' is acceptable. Never fail open for virus scanning on attachments: hold those until scanning succeeds.

FCM/APNs are best effort. Correctness comes from the history log: any app open, pull-to-refresh or periodic background sync catches up from the last history id.

Metadata and blobs are replicated across zones (3 replicas or erasure coding), so reads continue. MX hosts in other zones and regions are listed in DNS with MX priorities, so senders fail over automatically.