Prompt Caching, Fully Specified

Prompt caching is the single richest piece of mechanics OpenAI publishes for cost control, and it's also the easiest to get wrong quietly, because getting it wrong doesn't produce an error — it just produces a smaller discount than you expected, with nothing telling you why.

The shape of it

Caching turns on automatically once a prompt prefix clears a published minimum length — no code change required, no opt-in flag on older models. A routing step matches incoming requests against recently-seen prefixes using roughly the first slice of the prompt, and a matched prefix bills its cached portion at a steep discount against the ordinary input rate. That much is simple. What complicates it is that GPT-5.6 and later models charge for the act of writing a new prefix into the cache, at a markup over the plain input rate — every model before that generation wrote to the cache for free.

Why the write price changes the math

Because writes cost more on GPT-5.6+, caching there isn't automatically a win the way it was on older models — it only pays off once a prefix gets reused enough times for the accumulated read discount to outweigh the write markup. That's a real, computable break-even, not a vague "usually worth it": derived from the published write and read multipliers, not asserted, and it's exactly what the tool linked below exists to compute for a specific workload rather than leaving as an estimate.

The knobs that actually matter

prompt_cache_key gives the newer models' improved matching something stable to key against — required for reliable hits on GPT-5.6+, optional but still useful earlier. Explicit breakpoints let you mark exactly where a cacheable prefix ends, with a hard cap on how many new writes a single request can register and a separate, larger limit on how many prior breakpoints get considered when looking for a read match. Retention is short by default — the guide describes an ordinary in-memory lifetime measured in single-digit minutes of inactivity — with a much longer extended-retention option available by offloading the cached state off the hot path.

What actually invalidates a cache

A prefix that isn't byte-identical to a prior one doesn't match, full stop — a timestamp, a request id, or non-deterministic key ordering anywhere early in the prompt is enough to make every request unique from that point forward, silently. The table below and the invalidator list on this page cover the specific patterns worth checking your own prompts against before assuming a low hit rate is a routing problem rather than a prompt-construction one.