Design a Distributed Cache
Practice a distributed cache design with key partitioning, replication, eviction, invalidation, hot keys, cache stampedes, and degraded read behavior.
What you'll learn
Run the architecture drill
- How to partition cache keys and rebalance nodes without a full data shuffle.
- How eviction, TTL, invalidation, and replication change correctness behavior.
- How to protect the origin during cache failure or a hot-key burst.
A distributed cache is a latency and origin-protection system that must never become an unexamined second source of truth. Its real design problems are key placement, replication, eviction, invalidation, hot keys, stampedes, and how applications behave when the cache is slow or absent.
Start with the architecture drill, then follow the request path through scale, failure, and a defensible interview answer.
Make your assumptions and decision path easy to inspect.
ATOFF architecture reasoning canvas
Design a Distributed Cache: the architecture drill
Short answer
A distributed cache should reduce latency and origin load without becoming an accidental source of truth or a coordinated point of failure.
10× evolution
Shard hot-key ownership and control the pressure of rebuilding origin data.
01 Decision checkpoint
Design brief
The origin owns correctness; cache entries need freshness rules, versioning, and a defined application response when they are stale or absent.
02 Decision checkpoint
First request path
Partition by a stable hash, identify hot keys, and reserve replication or request coalescing for keys whose popularity breaks even distribution.
03 Decision checkpoint
Failure drill
Use jittered expiry, stale-while-revalidate, and single-flight regeneration so many misses do not recreate the same overload on the origin.
Say it in the interview
I would keep the origin authoritative, describe expiry and invalidation, then show how the system survives hot keys and a cache-wide miss.
- 1
Application
read key: Hash-ring key router
- 2
Hash-ring key router
route to partition: Cache partitions + replicas
- 3
Cache partitions + replicas
miss: Coalesced miss gate
- 4
Coalesced miss gate
one controlled fetch: Authoritative origin
- 5
Authoritative origin
populate: Cache partitions + replicas · change event: Invalidation stream
- 6
Write service
authoritative write: Authoritative origin
- 7
Invalidation stream
evict or refresh: Cache partitions + replicas
- Application flows to Hash-ring key router via read key.
- Hash-ring key router flows to Cache partitions + replicas via route to partition.
- Cache partitions + replicas flows to Coalesced miss gate via miss.
- Coalesced miss gate flows to Authoritative origin via one controlled fetch.
- Authoritative origin flows to Cache partitions + replicas via populate.
- Write service flows to Authoritative origin via authoritative write.
- Authoritative origin flows to Invalidation stream via change event.
- Invalidation stream flows to Cache partitions + replicas via evict or refresh.
Walk the design under pressure
Define the cache contract before the data structure
Clarify what values are cacheable, how stale data is tolerated, what keys identify them, whether writes invalidate or update cache entries, and whether a cache miss can safely reach the origin. A user profile snapshot and a payment balance may have very different freshness contracts.
The initial model can be cache-aside: the application reads the cache, fetches the authoritative origin on a miss, and populates a bounded TTL. It is simple but makes miss behavior, serialization, and invalidation the application's responsibility.
Partition keys to spread capacity without losing control
Hash-based routing spreads many independent keys over partitions and reduces data movement when nodes change. Virtual partitions help balance uneven hardware or gradual expansion. But hash distribution does not solve an individual hot key; a viral object can still overwhelm the one node assigned to it.
Replication can protect read availability and hot keys but consumes memory and complicates invalidation. Decide whether one replica is a performance copy, a failover copy, or both, and specify what applications do when replicas disagree briefly.
Eviction and expiry are workload decisions
Least-recently-used behavior may work for repeated interactive reads, while TTL ensures data eventually refreshes. Eviction is not an error by itself; the danger is many simultaneous misses that send a stampede to the origin. Use request coalescing, stale-while-revalidate, jittered expiry, or per-key locks when the origin cannot absorb that burst.
A cache outage should not cause every instance to retry aggressively. Circuit-break cache calls when it is unhealthy, cap origin fallback, and choose which reads can be rejected or served stale to protect correctness and core availability.
Staff-level insight: cache health must include the origin
At staff scope, cache hit rate alone is insufficient. Track origin load caused by misses, key-size distribution, evictions, hot partitions, stale-serving rate, invalidation lag, and the effectiveness of stampede controls. A cache can appear healthy while quietly moving expensive work to a database.
In an interview, describe evolution: begin with managed cache-aside, add partition routing when data exceeds one node, add hot-key replication and stampede protections after observing the workload, and keep a tested cache-bypass failure plan.
Keep this with you
Key takeaways
- A cache accelerates an authoritative origin; it should not silently become the only truth.
- Partitioning solves aggregate capacity, while hot keys and stampedes need separate protections.
- Cache failure behavior must protect the origin and preserve the correct user contract.
Practice aloud
Interview questions to explore
- 1.How do you prevent all clients from refilling an expired key at once?
- 2.What happens if the cache returns stale data after a critical update?
- 3.How does a hot key differ from a hot partition?
Common follow-ups
Frequently asked questions
Does consistent hashing solve cache rebalancing completely?
It reduces how many keys move when nodes change, but it does not by itself address uneven key sizes, hot keys, replication, or the migration process for in-flight traffic.
Should a cache write through to the database?
Write-through can simplify some consistency paths but adds write latency and cache dependency. Cache-aside is often simpler for read-heavy data; the best choice depends on the correctness and write workload.
Already an Elite member? Open the complete walkthrough.
Need the broader preparation context? Go back to Interview Preparation for behavioral readiness, question practice, and the larger AceTheOffer preparation framework.