Design a distributed counter for likes and views

The naive design runs UPDATE posts SET views = views + 1 WHERE id = ? on every view. On a viral video at 200k views/s, every update needs the same row lock, so they run one after another; a single row in MySQL or Postgres manages maybe a few thousand such updates per second before lock waits and replication lag explode. Redis INCR is much faster (100k+/s per key) but one key still lives on one shard and one CPU core, and all the network traffic for that item hits one node. The general fix is to aggregate before writing. Option one: sharded counters, where the count for item X is stored as N sub-counters (X#0..X#N-1); each increment picks a random shard and reads sum all N. Google's App Engine documentation popularised this pattern. Option two: pre-aggregate in the write path: app servers or a stream processor sum increments in memory for a few seconds and write one delta per item per interval. At 1M events/s across 50k active items, this cuts writes to ~10k/s. The cost is lag of a few seconds and possible loss of an in-memory window on crash unless the source is a replayable log like Kafka.

The thing that must be exactly right for likes is membership: a user has either liked a post or not, and must never be counted twice. Store this as a row keyed by (post_id, user_id). Because the key is unique, a retried request or a double tap is naturally idempotent. The count is derived data. Emit a delta event only when the state actually changes (like on a not-liked post, or unlike on a liked one), otherwise retries would inflate counts. Applying deltas to the count must also be idempotent: Kafka consumers can reprocess messages after a crash. Flink with checkpoints gives exactly-once state inside the job; to make the sink effectively-once, either write absolute totals (set count = 1,234,567, which is safe to repeat) or store the last applied offset per partition in the same transaction as the counts. Small drift can still creep in (bugs, lost events), so a periodic job recounts likes from the source table for active posts and fixes the counter. The UI uses the user's own like state from the likes table plus an optimistic +1, so the user sees a consistent picture even while the global count lags.

A view is a product definition, not a technical one: YouTube-style systems count a view only after a minimum watch time, and filter repeated views from the same viewer and bots. This matters because views often drive creator payouts and ad billing. So the pipeline has two tiers: a fast, approximate counter that shows on screen within seconds, and an exact, audited daily count computed in batch from raw events after fraud filtering. Unique viewers are a cardinality problem: storing every viewer ID per video is too much memory for billions of items. HyperLogLog estimates distinct counts using ~12 KB per counter in Redis (PFADD/PFCOUNT) with about 0.81% standard error, and two HLLs can be merged, so you can compute weekly uniques from daily ones. For exact uniques in creator analytics, run COUNT(DISTINCT) in the warehouse overnight. Explain both and when each is used; interviewers like candidates who treat 'view count' as a product and integrity question, not only a scaling one.

Flushing aggregates every 1 s gives near-real-time counts but 10× the DB writes of a 10 s window. For views, 10-30 s is fine. For likes, 2-5 s keeps the experience 'live' while cutting writes. Users mostly care about their own action, which the UI shows immediately.

Sharded counters are simple and synchronous but make reads N times more expensive and do not de-duplicate. Stream aggregation (Kafka + Flink) handles huge volume, dedupe and windows, but adds infrastructure and seconds of lag. Large platforms use stream aggregation for views and sometimes sharded counters for moderate-volume likes.

Showing '1.2M views' does not need exact numbers, so approximate online counters are fine. Money-related numbers (payouts, ad billing) must come from the reconciled batch pipeline. Making the online path exact would cost far more for no user benefit.

Keying by item_id sends a viral item's events to one partition and one Flink task. Detect hot keys and salt them (item#0..#15), aggregate partial sums in parallel, then combine with a second, small aggregation step.

5M reads/s for counts on every feed card. Batch reads (one MGET per feed page), cache hot items in Redis and in the feed service's local cache for a few seconds, and accept slightly stale numbers.

It restarts from the last checkpoint (e.g. every 30 s) and reprocesses Kafka from the checkpointed offsets. Because the sink writes absolute totals or tracks applied offsets, reprocessing does not double count. Counts on screen freeze for a minute, then catch up.

Count reads fall back to the counts table; the read service limits concurrency and serves 'no count' for non-critical surfaces if the DB is overloaded. The feed never fails because of counts.

Use a transactional outbox or CDC from the likes table (e.g. Debezium on the DB change log) so the event is derived from the committed write; otherwise the count drifts. The nightly recount corrects any remaining drift.