Hot keys, hot partitions and fan-out

Partitioning by key assumes keys get similar traffic. Real traffic follows a power law: a few keys get most of it. A hot key is one key with far more traffic than others (the score of an India vs Pakistan match, the iPhone listing at 12:00 on sale day, a celebrity's profile). A hot partition is a shard overloaded because of one hot key or many busy keys that hash to it. Adding more shards does not help, because one key always lives on one shard.

Every store has a per-partition ceiling. A single Redis node handles roughly 100,000 to a few hundred thousand simple operations per second on one main thread; a DynamoDB partition supports up to 3,000 read units and 1,000 write units per second; a Kafka partition is consumed by one consumer in a group. One key cannot exceed its partition's ceiling, however big the cluster.

Fan-out is the related problem of one action creating work for many keys. When a user with 50 million followers posts, fan-out on write (push) inserts the post into 50 million timelines; fan-out on read (pull) leaves timelines alone and merges recent posts from followed accounts at read time. Each moves cost to a different place, and big accounts break the pure push model.

1. 1Detect: track per-key or per-partition request rates (Redis --hotkeys or client-side sampling, DynamoDB CloudWatch Contributor Insights, Kafka per-partition lag). Alert when one partition's load is far above the average.
2. 2Hot reads, first fix: cache close to the caller. A small in-process cache (Caffeine, a local LRU) with a 1 to 5 second TTL on each app server means 200 servers send at most about 200 requests per TTL window to Redis instead of hundreds of thousands per second.
3. 3Hot reads, second fix: replicate the hot key. Store copies under key#1 ... key#N on different shards and let readers pick one at random; writes update all copies. Or serve it from a CDN if it is public, such as a match scorecard.
4. 4Request coalescing: when many requests miss the same key at once, let one fetch it and the others wait for that result (single-flight), so a cache miss does not become a stampede on the database.
5. 5Hot writes, split the key: a counter becomes N sub-counters (likes:post42:0 ... likes:post42:15), each write goes to a random one, and reads sum them. Write capacity grows N times; reads get slightly more expensive.
6. 6Hot writes, buffer and batch: aggregate increments in memory or in a stream for a second, then write one combined update. Ten thousand +1s become one +10,000.
7. 7Hot partitions from bad keys: avoid monotonic or low-cardinality partition keys (timestamp, date, country). Add a salt or bucket to spread load, for example (event_date, bucket 0-31), and query all buckets in parallel.
8. 8Fan-out: push for normal users (precompute timelines in Redis at write time), pull for celebrities above a follower threshold (merge their recent posts at read time), and skip pushing to inactive followers. This hybrid is what most large feeds use.
9. 9Protect the hot path anyway: rate-limit per key, shed low-priority work, and design queues so one hot key does not block a whole partition, for example by using a dedicated topic or partition for it.

# Split a hot counter across N sub-keys; read by summing
import random
N = 32

def incr_like(r, video_id: str, n: int = 1):
    r.incrby(f"likes:{video_id}:{random.randrange(N)}", n)

def get_likes(r, video_id: str) -> int:
    keys = [f"likes:{video_id}:{i}" for i in range(N)]
    # In Redis Cluster these keys hash to different slots on purpose, so read them
    # with a pipeline per node or a background job that stores the total.
    return sum(int(v or 0) for v in (r.get(k) for k in keys))