Design a real-time leaderboard

A Redis sorted set is a skiplist plus a hash map. Insert or update is O(log N), rank lookup is O(log N), and a range of the top K is O(log N + K). For 20M members, log N is about 25 steps, so each operation takes microseconds and one node handles over 100k ops/s. Memory is around 100 bytes per member, so 20M members is about 2 GB, comfortably on one node. A SQL alternative, SELECT COUNT(*) WHERE score > my_score, needs an index scan proportional to your rank, which is slow for users ranked millions deep and terrible at high read QPS. Limits appear in three places: a single board's write rate is capped by one Redis core (~100k-200k ops/s pipelined); very large boards (hundreds of millions, e.g. an all-time global game ranking) exceed comfortable memory on one node; and because Redis is in memory, you need a durable source of truth and a rebuild path. Say all three, then pick fixes only when the numbers demand them. For most contests one sorted set per board, with a replica for reads, is the right answer.

If one board outgrows a node, shard it. Sharding by user_id hash spreads load evenly but makes rank hard: your global rank is the sum of 'how many users beat me' across all shards, so a rank query fans out to every shard (ZCOUNT score +inf on each). With 10-20 shards that is acceptable for occasional reads, and top-N becomes a merge of each shard's top N. Sharding by score range instead puts users with scores 0-1,000 on shard A, 1,000-5,000 on B, and so on. Rank is then local rank plus the total size of all higher shards (cached counts), so a single query is enough. The downside is hot and uneven shards and users moving between shards when scores change, which needs a delete-then-add across nodes. A third option is approximate rank: keep exact sorted sets only for the top ~100k, and for everyone else maintain a histogram of scores in buckets; a user's rank is the count of users in higher buckets plus an estimate within their bucket. Users ranked 4,512,330 do not care whether it says 'about 4.5M'. Many games use exactly this.

Ties matter when money is involved. Redis orders equal scores lexicographically by member, which is arbitrary from the user's point of view. Define the tie-break rule explicitly with product: for example, the earlier team to reach that score wins, or the team created earlier wins, or ties share the rank and split the prize. If you need a tie-break in the sorted set, encode it into the score: composite = points × 10^k + (MAX_TS - timestamp), which still fits in a double's 53-bit integer precision if you size k carefully (fantasy points with one decimal and a seconds-level timestamp fit). For final results, do not trust the live Redis board. At the end of the contest, freeze writes, then compute final standings from the durable scores table with a deterministic SQL query (ORDER BY points DESC, tie_break ASC), write them as an immutable table and export to S3. Compare with the Redis view as a sanity check. Payouts read only the frozen standings, and every number is traceable to score events.

During a live match, millions of users refresh constantly. The top-100 list is identical for everyone in that contest, so cache it in the read service memory or a CDN-friendly endpoint with a 1-2 s TTL; it then costs one Redis call per second per server instead of tens of thousands. 'My rank' differs per user, but it is just one or two O(log N) calls; serve these from Redis read replicas so the primary keeps CPU for writes. Push rather than pull is also possible: send rank updates over WebSocket only to users whose rank changed significantly, but for fantasy contests polling every 30 s with caching is simpler and good enough. Friends leaderboards are small: fetch the friend list (hundreds), then ZSCORE them with a pipeline and sort in the service, instead of maintaining a sorted set per user.

A database is durable and supports complex queries, but rank queries scale with rank depth and do not survive 200k reads/s. Redis is fast and fits the access pattern. Use both: DB for truth, Redis as the serving index.

Exact ranks for everyone need one big set or a fan-out query. Approximate ranks via histograms are cheap and scale indefinitely. Use exact for the top segment where users compare closely and money depends on it; approximate for the long tail.

ZINCRBY per event is simple for games. For fantasy sports, one match event touches millions of teams, so batch recompute with parallel workers and pipelined ZADDs is the right model.

We accept a few seconds where some teams show the new score and others the old one. For the final result we freeze and recompute from truth, so correctness where it matters is guaranteed.

One Redis core limits one sorted set. Pipeline aggressively, shard by score range for extreme boards, or split huge contests into multiple sub-leaderboards that the product already shows separately.

A big match can have hundreds of thousands of contests, each with its own board. Spread board keys across the Redis cluster by hash slot; recompute contests in parallel; prioritise the largest and most viewed contests.

A replica is promoted (Redis Sentinel or Cluster). Any updates not yet replicated are recovered by replaying recent Kafka events, since updates are idempotent ZADDs of absolute scores.

Consumer lag grows and the leaderboard is stale. Autoscale updaters up to partition count; show 'last updated at' in the UI so users know. Final standings are unaffected because they come from the DB.

Stream the scores table into a new sorted set key with pipelined ZADDs (20M in a couple of minutes), then atomically switch an alias key or config pointer to it.