Server-Side Caching: The Hard Part Is Invalidation

Adding a cache is easy. Store a value, set an expiry, check for it before doing the expensive thing. An afternoon's work, and the response times drop immediately.

Then the interesting problems start. Someone updates a product, and the old price stays visible for an hour. A popular key expires under load, and four hundred requests hit the database simultaneously. A cached page leaks one user's data to another. The hit rate looks excellent, and the p99 latency hasn't moved.

None of those is caused by the caching layer being wrong. They're caused by the decisions around it, which is where this article focuses.

The rule

Cache as close to the user as correctness allows.

Every layer you move outward removes more work: fewer bytes over the network, fewer processes involved, less computation repeated. A response served from a CDN edge never touches your infrastructure. A response served from application memory never touches your database.

The constraint is correctness. The further out you cache, the less you know about who's asking and the staler the data can become. So the practical question for any given response is: what's the outermost layer that can serve this without being wrong?

The layers, from the outside in

HTTP and CDN caching is the largest available win and the most consistently underused. Get your Cache-Control headers right, and a CDN serves the response without involving your servers.

Cache-Control: public, max-age=60, s-maxage=3600, stale-while-revalidate=86400

Three separate instructions there. max-age governs browsers, s-maxage governs shared caches like your CDN, and stale-while-revalidate lets the CDN serve a slightly stale response immediately while it refreshes in the background. That last directive is the one people miss, and it removes most of the latency spike that expiry would otherwise cause.

Add ETag so conditional requests can return 304 with no body, and use private for anything user-specific so a shared cache never stores it.

Full-page and fragment caching store rendered output. Whole pages where content is identical for everyone, fragments where most of a page is shared, and a small part is personalised. Fragment caching is the more useful of the two in practice, because "identical for everyone" is rarer than it sounds once you have a logged-in header.

Application-level object caching is where most caching actually happens: computed values, serialised objects, API responses from third parties, anything expensive to produce and reused across requests. Redis or Valkey, and the licence situation there changed recently in a way worth knowing about, which I covered in the piece on server-side tools.

Query result caching needs a correction, because the standard advice is outdated. MySQL removed its built-in query cache in version 8.0, having deprecated it in 5.7, and PostgreSQL never had an equivalent. The reason is that a shared query cache invalidates on every write to any table involved, which made it a bottleneck under write load rather than a benefit. So query caching today means caching results in your application layer, deliberately, with keys and invalidation you control. For genuinely expensive aggregations, a materialised view refreshed on a schedule is usually the better tool.

Opcode caching isn't a decision any more. PHP ships OPcache enabled by default, and the JIT arrived in PHP 8. There's nothing to implement, and treating it as a technique to adopt dates any article that does.

Invalidation, which is the actual subject

The joke about cache invalidation being one of the two hard problems in computer science is repeated so often that people stop hearing it. It's accurate. Here's what it means in practice.

Time-based expiry is the simplest, and it's a bet: you're accepting up to one TTL of staleness in exchange for never having to think about it. Fine for content where being a few minutes behind is harmless. Not fine for prices, stock levels, or permissions.

Explicit invalidation deletes the key when the underlying data changes. Correct, and it requires you to know every key affected by every write, which is the part that quietly rots as the codebase grows. Someone adds a new write path six months later and doesn't know about the cache.

Key versioning avoids deletion entirely by building the version into the key:

product:1234:v7

Bump the version on write, and the old entry becomes unreachable, then expires on its own. No deletion logic, no risk of missing a key, and no stale reads. The cost is memory holding orphaned entries until they age out, which is usually a good trade.

Tag-based invalidation groups related keys so you can invalidate a whole category at once. Useful when one write affects many cached values, and several frameworks support it natively.

Then the three write strategies, which decide where the cache sits relative to your writes:

Cache-aside is the default: the application checks the cache, falls back to the database on a miss, and populates the cache itself. Simple, but a miss costs a full round trip.

Write-through updates cache and database together on every write. Consistent, slower writes.

Write-behind updates the cache immediately and the database asynchronously. Fast, and you can lose data if the process dies before the write lands. Use it deliberately or not at all.

The failure modes

The specific ways caching goes wrong, which are worth recognising before you meet them.

Cache stampede, also called the thundering herd. A popular key expires, and every concurrent request misses at once and hits the database. The database, which was comfortable a moment ago, falls over. The fixes: add jitter to your TTLs so keys don't expire in lockstep, use a lock so only one request recomputes while others wait, or refresh proactively before expiry. stale-while-revalidate solves this at the HTTP layer for free.

Caching user data in a shared cache. The most dangerous one, because it's a data leak rather than a performance bug. Any cache key for personalised content must include the user identity, and any HTTP response containing it must be marked private. A full-page cache in front of an authenticated page will eventually serve one person's account to another.

Unbounded growth. A cache without an eviction policy and a memory limit will consume everything available and then start failing in ways that look unrelated. Set maxmemory and choose an eviction policy deliberately.

Caching errors. A failed upstream call returning an error, cached with a long TTL, turns a transient outage into a persistent one. Cache negative results with short TTLs or not at all.

Key collisions and key sprawl. Adopt a naming convention early, something like entity:id:version, because a cache full of ad hoc keys is impossible to reason about or invalidate.

What caching hides

The part worth being honest about.

A cache in front of a slow query makes the slow query invisible. It's still slow, it still runs on every miss, and it will surface again the moment the hit rate drops or the cache is cleared during a deploy.

So before caching anything, check whether the underlying operation should be that expensive. An N+1 query pattern, a missing index, or an unnecessary join is a bug, and caching it is treating the symptom. Read the query plan first.

The right mental model: a cache avoids repeating work that's already reasonably fast. It doesn't fix work that's too slow.

The same applies to the SEO claim that circulates with caching advice. Speed is a genuine ranking factor and a modest one, functioning closer to a tiebreaker than a lever, which I set out in the piece on SEO for front-end developers. Cache because it improves the experience and reduces cost, not because you're expecting rankings to move.

Measuring it

Three numbers, and the first one is the least useful on its own.

Hit rate tells you what proportion of lookups found something. High is good, and high alone means nothing: a 99 per cent hit rate on data nobody requests is meaningless. Read it alongside volume.

Latency percentiles, not averages. A cache improves the average dramatically while the p99 stays exactly where it was, because the p99 is the misses. Users experience the p99. Track it.

Database load, which is the point of most of this. If your cache hit rate is excellent and query volume hasn't dropped, you're caching things nobody asks for twice.

Worth also tracking eviction rate, since a cache evicting heavily is too small for its working set, and that shows up as a mysteriously falling hit rate rather than as an error.

Where to start

In order of return on effort.

Set proper HTTP cache headers on static assets and anything public. Costs nothing, removes the most traffic, and works before any request reaches your application.

Put a CDN in front of everything, with s-maxage and stale-while-revalidate on the responses that allow it.

Cache the expensive computed values you can identify from your slow query log or traces. Start with the worst offender rather than caching broadly.

Add fragment caching to the shared parts of pages that are otherwise personalised.

Then measure again, because the bottleneck will have moved and the next thing to cache is rarely the one you assumed.

The short version

Cache as far out as correctness permits, because every layer outward removes more work than the one inside it.

Invalidation is the real problem, and key versioning avoids most of it by making stale entries unreachable rather than requiring you to find and delete them.

Know the failure modes before you meet them: stampedes, user data in shared caches, unbounded growth, cached errors.

Fix slow queries before caching them, since a cache in front of a bad query hides a bug rather than solving it.

Measure p99 rather than the average, because the average reflects what your cache improved, while p99 reflects what your users actually experience.