What is Prompt caching?

Reusing previously processed prompt tokens to cut cost and latency on repeated context — many APIs discount cached input heavily.

Prompt caching lets an API provider reuse the processed form of prompt text it has already seen, instead of recomputing it on every request. When your requests share a long, unchanging prefix (a system prompt, tool definitions, a big document, or earlier conversation turns), the provider stores the computed state and applies it directly next time. Cached input tokens are billed at a steep discount: Anthropic charges about 10 percent of the normal input price for cache reads, and OpenAI about 25 to 50 percent depending on the model. Latency also drops because the model skips reprocessing. Caches expire after minutes to an hour unless refreshed, and only exact prefix matches count, so prompt structure matters: put stable content first and variable content last.

Example

A coding assistant sends the same 30,000-token system prompt and tool definitions with every message. With caching, those tokens cost around a tenth of the normal rate after the first request, cutting the bill dramatically for long sessions.

Why it matters

For chatbots, agents, and any app with long repeated context, caching can cut API costs several-fold. When comparing providers, check cache pricing and expiry, not just the headline per-token rate. Browse the AI tools directory or the model leaderboard to put it into practice.