Caching
In one line
A cache keeps a copy of frequently used data in fast storage close to where it is needed, trading some freshness and extra complexity for much lower latency and less load on the source.
What it is
A cache is a fast, usually in-memory store that holds copies of data whose original lives somewhere slower, like a database, another service or disk. Reading from Redis takes well under a millisecond on a local network, while a database query might take 5 to 50 ms.
Caches appear at every layer: the browser, a CDN at the edge, an in-process cache inside the application, a shared distributed cache like Redis or Memcached, and the database's own buffer pool. The key design questions are what to cache, how to keep it fresh (writes and expiry), and what to evict when memory is full.
Why interviewers ask
Caching is the first tool for read-heavy systems, so it shows up in nearly every design interview. Interviewers probe whether you understand invalidation, stale data, eviction policies and failure modes such as cache stampedes, not just 'put Redis in front of it'.
How it works
- 1Cache-aside (lazy loading), the most common pattern: the app checks the cache; on a hit it returns the value; on a miss it reads the database, writes the result to the cache with a TTL, and returns it.
- 2Write-through: every write goes to the cache and the database together, so the cache is always populated and fresh, at the cost of slower writes and caching data that may never be read.
- 3Write-back (write-behind): writes go to the cache and are flushed to the database later in batches. Very fast writes, but data can be lost if the cache node dies before flushing.
- 4Invalidation on writes: with cache-aside, update the database then delete the cache key, so the next read reloads fresh data. Deleting is safer than updating the cached value, which can race with concurrent writers.
- 5Expiry: a TTL (for example 60 seconds) bounds how stale data can get even if an invalidation is missed.
- 6Eviction: when memory is full the cache drops entries by policy: LRU (least recently used, the usual default), LFU (least frequently used) or FIFO.
- 7Measure the hit rate. A 90% hit rate means only 1 in 10 reads reaches the database.
Worked example · Cache-aside for a product page with a 60-second TTL
- t = 0 s: GET product 42. Redis has no key product:42 (miss). The app queries the database (20 ms), stores the row in Redis with TTL 60 s, returns it.
- t = 5 s: GET product 42. Redis hit, returns in under 1 ms. The database is not touched.
- t = 30 s: an admin changes the price. The app updates the database, then deletes product:42 from Redis.
- t = 31 s: GET product 42 misses again, reloads the new price from the database and caches it until t = 91 s.
- If the delete at t = 30 s had failed, users would see the old price for at most the remaining TTL, until about t = 60 s. That bound is why you keep a TTL even with explicit invalidation.
Picture it
A cache is the notepad on your desk with the phone numbers you call most. Looking there is instant compared with searching the company directory, but if someone changes their number and you do not update the notepad, you dial the old one.
Key patterns
Cache-aside
When: General read-heavy data, default choice
How: Read cache, on miss load from DB and populate with TTL; on write update DB then delete the key.
Write-through
When: Data that is read right after it is written and must be fresh
How: Write to the cache and database in the same operation path; reads always hit a fresh cache.
Write-back
When: Very high write rates where small data loss is acceptable (counters, metrics)
How: Write to the cache, flush to the database asynchronously in batches.
Stampede protection
When: A hot key expires and thousands of requests miss at once
How: Use a lock or request coalescing so one request rebuilds the value, add random jitter to TTLs, or refresh hot keys before they expire.
Multi-level caching
When: Extreme read rates on a few keys
How: Small in-process cache (seconds) in front of Redis (minutes) in front of the database.
Real-world use
Facebook and Memcached
Facebook ran one of the largest Memcached deployments, using a look-aside cache in front of MySQL and leases to prevent stale sets and thundering herds, as described in its 'Scaling Memcache at Facebook' paper.
CDNs (Cloudflare, Akamai, CloudFront)
Cache images, scripts and even whole pages at edge locations near users, controlled by HTTP Cache-Control headers, so most requests never reach the origin.
Redis as a shared cache
Widely used for sessions, user profiles, leaderboards and API responses, with TTLs and eviction policies such as allkeys-lru configured per use case.
Database buffer pools
PostgreSQL's shared buffers and MySQL InnoDB's buffer pool cache hot data pages in memory so repeated queries avoid disk reads.
In a project
A news site's article API serves 20,000 reads per second at peak and the PostgreSQL primary is at 90% CPU, with p99 latency of 800 ms.
- Measure: 95% of reads hit the top 1,000 articles, so the working set is small enough for a cache.
- Add cache-aside with Redis: key article:{id}, TTL 5 minutes, delete the key whenever an editor publishes a change.
- Prevent stampedes on breaking news: when a hot key misses, only one request (holding a short Redis lock) rebuilds it while others briefly serve the old value or wait.
- Put the rendered HTML and images behind a CDN with a 60-second Cache-Control max-age.
- Dashboard the cache hit rate, evictions and database QPS.
Impact: The cache hit rate reaches 97%, database reads drop from 20,000 to about 600 per second, database CPU falls to 25%, and p99 latency drops from 800 ms to 40 ms.
Trade-offs
Use it when
- Reads far outnumber writes and the same items are read repeatedly
- Results are expensive to compute (aggregations, rendered pages, remote API calls)
- Slightly stale data is acceptable for a bounded time
Avoid it when
- Data must always be read-your-latest-write and correct, such as account balances during a payment, unless reads go to the source
- Access is spread evenly over a huge keyspace, so the hit rate would be low
- The source is already fast enough; a cache adds another system to keep consistent
Common mistakes
- Saying 'add a cache' without explaining invalidation, TTL and what happens on a miss storm.
- Updating the cached value on write instead of deleting it, which can leave stale data when two writers race.
- Not having a TTL at all, so one missed invalidation leaves data wrong forever.
- Ignoring the cold start: after a cache restart all traffic hits the database at once.
- Caching per-user or rarely repeated data and wondering why the hit rate is low.
Interview questions
Quick revision
- Cache-aside: read cache, miss -> load DB -> set with TTL. Write: update DB, delete key.
- Write-through: fresh but slower writes. Write-back: fast writes, risk of loss.
- Always set a TTL as a staleness bound.
- Eviction: LRU default, LFU for stable popularity.
- Stampede fixes: lock or single-flight, TTL jitter, early refresh.
- Hit rate is the key metric; misses are what the DB sees.
- Layers: browser, CDN, in-process, Redis/Memcached, DB buffer pool.
Check yourself
1. In cache-aside, what should the application usually do after updating a row in the database?
2. Which write strategy risks losing data if the cache node crashes?
3. A popular key expires and 5,000 requests hit the database at the same moment. What is this called?
4. Which eviction policy removes the item that has not been accessed for the longest time?
See it in a full design
These walkthroughs use this lesson.
Saved on this device. Create a free account to keep it on every device.
Practise this with Sam
A real mock interview, out loud, with an honest scorecard. Free to start.