Every request to a language model pays to read the whole prompt again. For a chatbot with a two-line system prompt that is nothing. For an assistant that sends 12,000 tokens of instructions, tool definitions and policy documents before the user has typed a word, it is most of the bill, and it is the same 12,000 tokens every single time.
Prompt caching is the providers' answer: keep the processed form of a prompt prefix for a few minutes, and charge about a tenth of the input price when the next request starts with exactly the same tokens. All three big providers do it. They disagree on nearly every detail that decides whether it saves you anything: how you turn it on, how short a prefix can be, how long the cache lives, and whether writing to it costs extra.
Below are those rules as they stand in September 2026, taken from each provider's own documentation, and the arithmetic of when caching pays and when it quietly costs more than doing nothing.
What a prompt cache actually stores
When a model reads a prompt it turns every token into internal state that the following tokens depend on. Caching stores that state for a prefix of the prompt, so a later request starting with the same tokens can skip the work. That dependency is also why the matching rule is so strict: the state for token 5,000 depends on all 4,999 before it, so a cache can only be reused up to the first token that differs. Anthropic's documentation puts it plainly, saying hits "require 100% identical prompt segments, including all text and images up to and including the block marked with cache control".
Two things follow. Order matters more than content: the same ten blocks in a different order are a different prompt. And a change near the top is far more expensive than a change near the bottom, because it discards everything after it.
All three providers put the stable material at the front by construction. Anthropic documents the prefix order as tools, then system, then messages. OpenAI caches "the model's full rendered context", tool definitions and conversation history included. Google describes an explicit cache as "a prefix to the prompt". So tool definitions and instructions naturally sit where you want them.
The rules, provider by provider
| Anthropic | OpenAI | Google Gemini | |
|---|---|---|---|
| Turned on by | cache_control, per block or once at the top level | nothing, it is automatic | implicit by default; explicit via cachedContents |
| Minimum prefix | 512 (Opus 5), 1,024 (Sonnet 5), 4,096 (Haiku 4.5) | 1,024 on GPT-5.6 and later | 4,096 on 3.x Flash and 3.1 Pro |
| Lifetime | 5 minutes, refreshed free on each hit, or 1 hour | 30 minutes after the last write or reuse | explicit: 1 hour by default, any TTL you set |
| Write cost | 1.25x input, or 2x for the 1-hour cache | 1.25x on GPT-5.6 and later, none before | none for implicit; explicit pays hourly storage |
| Read cost | 0.1x input | 0.1x on GPT-5.6 and later | $0.20 against $2.00 input on 3.1 Pro |
Anthropic
Prompt caching on Anthropic is opt-in. Either you add one cache_control field at the top level and the API puts the breakpoint on the last cacheable block, moving it forward as the conversation grows, or you mark up to four blocks yourself. The explicit form is for prompts whose sections change at different speeds: tools that never change, a document that changes daily, a conversation that changes every turn.
The default entry lives five minutes and every hit refreshes it for free. The one-hour option costs twice the input price to write against 1.25x for the five-minute one, and reads are a tenth of the input price either way. On Claude Sonnet 5 that is $2.50 per million tokens to write, $0.20 to read and $2.00 to send uncached, per the pricing page.
Two details catch people out. The lifetime is measured from the start of the request, so a response that streams for four minutes leaves about a minute of usable cache. And "a cache entry only becomes available after the first response begins", so twenty requests fired in parallel from a cold start all miss.
Prompts under the minimum are processed without caching "and no error is returned". The only way to notice is cache_creation_input_tokens and cache_read_input_tokens both coming back zero.
OpenAI
OpenAI caches automatically on supported models, with nothing to mark up. On GPT-5.6 and later the minimum prefix is 1,024 tokens, an entry lasts "30 minutes after its most recent write or reuse", writes cost 1.25x the input rate and reads 0.1x. On earlier models such as GPT-5.4 mini there is no write charge at all, and retention is either in memory, typically "5 to 10 minutes of inactivity, up to one hour", or an extended 24-hour option on a listed set of models.
The part that is easy to miss is routing. A request goes to a machine chosen from a hash of the start of the prompt, and the cache lives on that machine. OpenAI's guidance is to aim for about 15 requests a minute per prompt_cache_key; above that, traffic can overflow to machines that do not hold the entry. The key is there so you can group traffic deliberately, per customer for instance, or spread one very busy prefix. Cached tokens come back in usage.input_tokens_details.cached_tokens.
Google Gemini
Gemini has two modes. Implicit caching is on by default for Gemini 2.5 and newer, with a 4,096-token minimum on the current 3.x models. Google says it will "automatically pass on cost savings if your request hits caches", while also stating there is no cost-saving guarantee. Its two tips are the ones everybody gives: put large, common content at the beginning, and send requests with a similar prefix close together in time.
Explicit caching is the other mode. You store content once as a cachedContents object, reference it by name, and pay hourly storage on top of discounted reads. The default lifetime is one hour and there are no minimum or maximum bounds on it. On Google's pricing page, storage is $4.50 per million tokens an hour on Gemini 3.1 Pro and $0.50 on Gemini 3.8 Flash. A 12,000-token prefix kept for a full day therefore costs about $1.30 on 3.1 Pro and $0.14 on Flash: nothing next to a busy prefix's savings, pure loss for a quiet one.
The break-even hit rate
With no write premium, caching cannot lose. A miss costs what it would have cost anyway and a hit costs a tenth. With a premium, every miss that writes costs more than not caching at all, and you need enough hits to pay for those writes.
If every miss writes the cache, the prefix costs h × 0.1 + (1 − h) × w of its uncached price, where h is the hit rate and w the write multiplier. Set that equal to 1 and solve:
break-even h = (w − 1) / (w − 0.1)
5-minute write, w = 1.25: 0.25 / 1.15 = 21.7%
1-hour write, w = 2.00: 1.00 / 1.90 = 52.6%That matches the rule of thumb in Anthropic's own pricing page: a five-minute write pays for itself after one read, a one-hour write after two.
quick check
You switch to Anthropic's 1-hour cache, which writes at 2x the input price and reads at 0.1x. If every miss writes the cache, what hit rate do you need before caching saves anything?
The prefix costs h × 0.1 + (1 − h) × 2 of its uncached price, which equals 1 when h = 1 / 1.9, about 52.6%. The 22% figure belongs to the 5-minute cache, which writes at 1.25x.
The one-hour cache is for traffic with gaps longer than five minutes but shorter than an hour: an internal tool used through the working day, an agent waiting on a slow tool, a queue of documents handled one at a time. If requests arrive every few seconds anyway, the five-minute entry is refreshed by every hit and never expires, and paying double to write buys nothing.
A worked example
A support assistant on Claude Sonnet 5 takes 20,000 requests a day. Each sends a 12,000-token stable prefix (instructions, tool definitions, the refund and shipping policies) plus 800 tokens that change per request, and gets 400 tokens back.
Uncached, a request costs $0.0256 of input and $0.0040 of output. That is $592 a day, or $17,760 over a 30-day month.
With the five-minute cache and 90% of requests hitting it, 18,000 requests read the prefix at $0.20 per million and 2,000 write it at $2.50 per million. The month comes to $6,456, a saving of 63.6%. The one-hour cache at the same hit rate costs $7,536, because its writes are dearer and, at this volume, the longer lifetime wins no extra hits.
Two things in that chart matter more than the headline saving. Even at 99% hits the bill only falls to about $4,966, because output and the 800 variable tokens are never discounted. And at 20% hits the same code costs $18,048, more than doing nothing at all. That is where a prompt with a clock at the top ends up, and no API response will tell you.
Work out your own numbers
The defaults are the support assistant above. Writes follow the misses until you edit them; lower them if some misses never write, for instance when a burst of parallel requests misses before the first write lands.
calculator · list prices per 1M tokens, checked 12 Sep 2026
What prompt caching saves on your traffic
where the cached bill goes, a month
Cache reads are priced at each provider's cached-input rate. Writes are 1.25x input on Anthropic's 5-minute cache and on OpenAI's GPT-5.6 and later, 2x on Anthropic's 1-hour cache, and free on GPT-5.4 mini and Gemini's implicit cache. Gemini hits are priced at the listed context-caching rate; Google does not guarantee a hit. Explicit Gemini caches also charge hourly storage, which this leaves out.
Ordering the prompt for cache hits
Most of the saving is decided by how the prompt is assembled, not by a setting:
- Stable first, variable last. Tool definitions, then instructions, then reference documents, then the conversation, then the new message. OpenAI's guide says the same thing: "Put stable developer instructions and shared reference material first".
- Nothing per-request in the system prompt. The time, the user's name, a request ID, a feature flag. Put them in the final user message, or at least after the breakpoint. A timestamp to the second in the first line guarantees a 0% hit rate.
- Serialise deterministically. Anthropic warns that some languages, Go and Swift among them, randomise JSON key order, which breaks caches. Sort keys and build the tool list in a fixed order.
- Do not toggle things mid-conversation. Changing tool definitions invalidates Anthropic's whole cache; adding or removing an image, or changing
tool_choice, invalidates the message blocks. - Keep history append-only. Summarising or trimming old turns rewrites the middle of the prompt, so everything after the edit becomes a write. Do it rarely, and in big steps rather than every turn.
- Warm the cache before a burst. Since an entry only appears once the first response has begun, send one request and wait for it before fanning out twenty.
- Group tenants on purpose. A prefix shared by all customers caches far better than one per customer. If each customer needs their own documents, put the shared part first and theirs after it.
Measure it, then trust it
Every provider reports cached tokens per request: cache_read_input_tokens and cache_creation_input_tokens on Anthropic, cached_tokens on OpenAI, and a cached-token count in Gemini's usage metadata. Log them on every call next to the model and the feature, and chart the hit rate. It is the number that tells you a deploy broke the prefix, and nothing else will until the invoice arrives. What to log around an LLM covers the rest of that log line.
Then put the real hit rate into your per-user maths. Caching usually moves the cost of an AI feature further than swapping models does, which is why it is the first lever in LLM cost per user. Voice agents feel it most, because they resend the whole prompt on every reply: the per-minute version of this arithmetic is in what an AI phone agent costs per minute.
When caching does not help
- Short prompts. Under the minimum nothing is cached, so a 3,000-token prompt will never hit on Claude Haiku 4.5 or on Gemini's 3.x models.
- Output-heavy work. Caching only discounts input. If most of your bill is generated tokens, long drafts or code, the saving is small however good the hit rate.
- Quiet, bursty traffic. A tool used a few times an hour misses most of the time. On a 1.25x write that is roughly break-even; on a 2x write it is a loss.
- Prefixes that differ per user. These only hit when the same user comes back inside the lifetime, which for a five-minute entry means during one session.
questions people ask
How does prompt caching work?
The provider stores the processed form of the start of your prompt. When a later request begins with exactly the same tokens, those tokens are billed at a discount, usually a tenth of the input price, and processed faster. Any difference ends the match at that point.
How much does prompt caching save?
It depends on how much of the prompt is stable and how often it hits. In the worked example, a 12,000-token prefix at a 90% hit rate on Claude Sonnet 5 cut the monthly bill by 63.6%. Output and per-request input are never discounted, so they set a floor.
What is the minimum prompt length for prompt caching?
On Anthropic it is 512 tokens for Claude Opus 5, 1,024 for Sonnet 5 and 4,096 for Haiku 4.5. OpenAI's is 1,024 on GPT-5.6 and later, and Gemini needs 4,096 on its 3.x models. Shorter prompts are processed normally, without caching and without an error.
How long does a prompt cache last?
Anthropic's lasts five minutes by default, refreshed by each hit, or an hour at a higher write price. OpenAI's lasts 30 minutes after the last write or reuse on GPT-5.6 and later. Gemini's explicit caches default to an hour and can be set to any lifetime, with storage billed hourly.
Is OpenAI prompt caching automatic?
Yes. It is enabled by default on supported models and needs no code change. On GPT-5.6 and later, writes cost 1.25x the input rate and reads 0.1x, and the usage data reports cached tokens per request.
Why is my cache hit rate low?
Usually something near the top of the prompt changes every request: a timestamp, a user name, unsorted JSON. The other causes are traffic gaps longer than the cache lifetime, parallel requests sent before the first one wrote the entry, and prompts under the minimum length.
The short version
Prompt caching bills a repeated prefix at about a tenth of the input price on all three providers. It only works when the prefix is identical from the first token, so order the prompt with the stable parts first and keep anything per-request out of the system prompt and the tool definitions.
Where writes cost extra, caching has a break-even hit rate: about 22% for a 1.25x write, about 53% for a 2x write. Below that line you are paying more than if you had never turned it on. Log cached tokens on every call, watch the hit rate like any other production metric, and reach for the one-hour cache only when your traffic really does have gaps longer than five minutes.