Prompt caching

Prompt caching is a model provider feature that stores the processed beginning of a prompt, so a later request that starts with exactly the same text can reuse it. Reused input is billed at a lower rate than fresh input and is usually processed faster.

What it is

The cache is a prefix match. The longest identical stretch at the start of a request is read from the cache; one changed character ends the match, and everything after it is processed and billed normally. Caches expire after a short time, often minutes, and a cache built for one model can't be used by another.

Providers differ in the details. Anthropic uses explicit cache markers in the request and charges for cache writes (prompt caching docs); OpenAI caches long prompts automatically. Check your provider's current pricing.

Why it matters

An agent resends its conversation on every turn of a tool loop, and each call's input is the previous call's input plus a little more. That's the pattern a prefix cache rewards. Volatile content early in the prompt, such as a timestamp, breaks the cache every time.

In AI CodeOps

Sessions that run on Claude models use prompt caching. The system prompt is cached as its own entry, and a marker at the end of the conversation moves forward each turn, so a step's tool loop reads its history from cache. Recorded cost prices cache writes and cache reads at their own rates, not at the full input price.

On the Usage page, open a model call's token count to see its split into cache reads and cache writes. The Learn guide goes deeper.