What Prompt Caching Actually Is

Prompt caching is a billing and routing feature, not a magic model upgrade. Anthropic stores portions of your prompt on their infrastructure for a short window. When a new request arrives with matching content at the start of the prompt, the cached portion is served from storage instead of being reprocessed token by token. You pay a small premium to write to the cache and roughly a tenth of the normal input price to read from it.

Under the hood, the cache is prefix-based. Anthropic looks at the leading sequence of tokens in your request and checks if it has seen that exact sequence recently. If the prefix matches a cached entry, the matching tokens are billed as a cache hit. Everything after the cache breakpoint is processed normally.

The important boundaries to internalize:

• Caching happens at the prefix level, not by string matching. Two requests with byte-identical text will hit, two requests that differ in a single character before the breakpoint will not.
• The cache has a short lifetime, typically around five minutes of inactivity, and refreshes on every hit. This is a working set cache, not a long-term store.
• There is a minimum cacheable length per breakpoint. In the current version this sits at 1024 tokens for Sonnet-class models and 2048 tokens for larger Opus-class models. Anything shorter is silently not cached.
• You can place up to four breakpoints in a single request, which lets you cache several distinct sections independently.

Strip away the marketing and the value proposition is straightforward. If you send the same large system prompt or tool definitions on every call, you are paying to re-tokenize and re-process the same content dozens of times per session. Caching collapses that repeated work into a single write.

Setup and Authentication

Prompt caching sits behind a beta header in the current API version, so the first thing to check is that your SDK or HTTP client is passing the right flag.

For the Python SDK at the current version, you import the beta messages wrapper:

from anthropic import Anthropic

client = Anthropic()

The cache control blocks live in the request itself rather than as a separate client option, so authentication is the same as any other Anthropic call. A standard API key from the console works. If you are already running requests against Claude, you are most of the way there.

If you are calling the raw REST endpoint, add the header anthropic-beta with the value prompt-caching-2024-07-31 (or the version Anthropic lists as current at the time you read this). The feature has been in beta for a while, and Anthropic occasionally ships a new beta string when they revise the underlying behavior. Check the release notes before pinning the header in production.

Two things to verify before you start:

1. Your account has prompt caching enabled. It is on by default for most accounts in the current version, but enterprise and org-level configurations sometimes disable beta features.
2. The model you are using supports caching. As of writing, the Sonnet 3.5 family and Opus 3 family both support it. Haiku support has been added in newer versions of the API and is worth confirming against the live model list if Haiku matters to you.

You do not need a separate cache key, a separate endpoint, or a new billing line. Everything funnels through the same messages interface, which is one of the cleaner parts of the design.

First Working Example

Here is a minimal end-to-end call that writes a cache entry on the first request and reads it on the second. The cache_control block on the system message marks that section as cacheable.

First call, which writes to cache:

response = client.beta.messages.create( model=“claude-3-5-sonnet-latest”, max_tokens=1024, system=[ { “type”: “text”, “text”: “You are an assistant that answers questions about a 200-page internal engineering handbook. ” * 50, }, { “type”: “text”, “text”: “Always answer in a structured format with headings.”, “cache_control”: {“type”: “ephemeral”}, }, ], messages=[{“role”: “user”, “content”: “What is the deployment checklist?”}], ) print(response.usage)

The first response will show input_tokens for the full prompt plus cache_creation_input_tokens for the section marked with cache_control. The output will also include a small extra charge for writing to the cache, in the range of roughly 25 percent above the base input price for that portion.

Second call, sent within the cache window, which reads from cache:

response = client.beta.messages.create( model=“claude-3-5-sonnet-latest”, max_tokens=1024, system=[ { “type”: “text”, “text”: “You are an assistant that answers questions about a 200-page internal engineering handbook. ” * 50, }, { “type”: “text”, “text”: “Always answer in a structured format with headings.”, “cache_control”: {“type”: “ephemeral”}, }, ], messages=[{“role”: “user”, “content”: “What is the on-call rotation policy?”}], ) print(response.usage)

The usage object on this call will show cache_read_input_tokens rather than cache_creation_input_tokens for that segment. That is the read, billed at roughly 10 percent of the standard input token rate. Your user message changes between calls, which is fine, because the cache is on the system prefix.

If you see cache_read_input_tokens populated, the feature is working. If you see cache_creation_input_tokens on every call, the cache has expired between requests or the prefix is drifting slightly.

Key Settings That Matter

The single most important dial is where you place the breakpoint. Two practical patterns to know:

Pattern one, cache the system prompt only. Put cache_control on the final system text block. This is the lowest-risk setup and works well for RAG-style applications where the system prompt carries a long, stable instruction set.

Pattern two, cache the system prompt plus the tool definitions. When you pass a tools array, each tool block can also accept cache_control. This pays off heavily for agents that ship a large tool schema, because the schema can run into the thousands of tokens and gets re-sent on every step.

Pattern three, cache the system prompt and the leading conversation turns. This is useful for chat applications where the early messages rarely change. The cache boundary falls right before the most recent user message, and only that final turn is billed at full price.

Other settings that matter and that people routinely ignore:

• The min token floor. If your cacheable section is below the model’s minimum, the cache_control directive is ignored. Measure your system prompt and tool schema in tokens, not characters, and aim for well above the floor to leave room for growth.
• Number of breakpoints. The current version caps you at four per request. If you need to cache a system prompt, a tool schema, and a long history, you can do it, but the math has to add up.
• Model selection. Caching is per-model. A cache written on Sonnet does not transfer to Opus, so pick a model and stick to it across the session.
• TTL behavior. The five-minute window is not a knob you can set. The only way to keep a cache warm is to hit it regularly. If your workload is bursty with long gaps, caching will not help.

A common mistake is to place cache_control on the very first content block. That works, but it forces the entire prefix to be the cache key, including any variable parts you forgot about. Putting the breakpoint as late as possible in the stable content gives you a larger cache hit rate.

Where It Shines

The strongest returns show up in three patterns.

Long, stable system prompts for RAG and document Q&A. If you are stuffing 50k tokens of reference material into the system prompt and answering 100 questions against it, the reference material only needs to be cached once. Subsequent questions read the bulk of the input from cache, which is the difference between a feasible product and a non-starter on cost.

Agent loops with large tool schemas. An agent that uses 15 tools, each with a detailed JSON schema, can easily cross 3k tokens of tool definitions per step. A 10-step agent run caches the schema on step one and reads it on steps two through ten. The savings compound with step count.

Chat applications with deep history. Customer support and tutoring apps where the first 20 turns of conversation stay on screen for the entire session. Caching the first 18 turns and only paying full price for the last two is a meaningful reduction in input cost.

In all three cases, the common factor is a large stable prefix and a small variable suffix. That is the geometry prompt caching is built for.

Where It Fails

Prompt caching is the wrong tool in several situations.

Short prompts. If your total input is under a few thousand tokens and the cacheable portion is below the model’s minimum, you get no benefit and you still pay a small write surcharge on the first call. Break-even is usually around request two or three on the same prefix.

Highly variable prefixes. If your system prompt includes timestamps, request IDs, or other tokens that change every call, the cache never hits. Caching works only on byte-identical prefixes, not semantically similar ones.

Bursty workloads. The five-minute window means a cache built during a busy hour is gone by the next morning. If your traffic pattern is spiky with long quiet stretches, the cache will expire before the next burst.

Multi-model orchestration. If you route the same prompt through different models for evaluation or A/B testing, you are paying the write cost on each model. The cost math gets worse, not better.

Cross-region or cross-account scenarios. Caches are scoped to a single organization. If you fan out across accounts for any reason, you lose the cache. This is rarely a problem in single-tenant apps but bites in consulting and platform contexts.

It also does not help with output cost. Caching only reduces the input bill. If your outputs are long and your inputs are short, prompt caching is the wrong lever.

Practical Workflow Pattern

A workflow that works in production looks like this.

Step one, audit your prompt. Tokenize your typical system prompt and tool definitions. Anything above the model’s minimum and stable across requests is a candidate for caching. Anything that changes every call is not.

Step two, place the breakpoint late. Put cache_control on the last stable block in the system and on the tools array if relevant. Avoid placing it on blocks that include runtime data.

Step three, measure before and after. Run a few hundred real requests with caching off and with caching on, and compare the cache_creation_input_tokens and cache_read_input_tokens fields in the usage object. Most teams see a 50 to 90 percent reduction in input cost on cached prefixes, depending on prefix size and hit rate.

Step four, log hit rate. Track the ratio of cache_read_input_tokens to total input tokens over time. A healthy number is in the 80 percent range for steady workloads. A low number means your prefix is drifting or your request rate is too sparse.

Step five, alert on cache misses. If your hit rate drops suddenly, something changed in your prompt construction. Treat it as a regression and investigate.

Step six, plan for cache invalidation. If you ship a new version of the system prompt, expect every cache entry to invalidate. The first request after the change pays the write cost. Plan releases accordingly so the cost spike is not a surprise.

Step seven, set the right model. Stick to one model per cacheable workflow. If you need to compare models, run them in parallel sessions rather than routing the same prefix through multiple models, because each model writes its own cache.

If you follow this pattern, the feature is a quiet, durable cost reduction. The most common failure mode is overengineering the cache layout when the basic pattern of “cache the long stable prefix” would have worked.

If you’re building with Claude or Codex right now, grab the free Working With Claude field guide. Thirty-two pages on the full ecosystem, Claude Code in depth, and how to roll agents out properly. Get the free guide.