Is prompt_cache_key Required on GPT-5.6?

The mistake

Treating prompt_cache_key as required for cache hits on GPT-5.6 and later models — adding keys everywhere, splitting traffic across keys to stay under a per-key rate, and blaming a missing key when the cache discount comes in low. This page used to recommend exactly that. OpenAI's prompt-caching guide now says the opposite for these models.

Why this happens

On models before GPT-5.6, a stable key genuinely did help: it steered related requests to the same machine, and OpenAI suggests keeping each key to about 15 requests a minute across all its prefixes. When the GPT-5.6 caching changes arrived, it was natural to read the key as even more important. On GPT-5.6 and later, OpenAI handles cache routing automatically. The key is no longer needed to optimize caching at all — it now serves one purpose: keeping cache accounting separate between your customers or users, which also stops one user probing whether another's content was cached.

Why it matters

If your hit rate on a GPT-5.6 or GPT-6 model is low, the key is almost never the cause, and debugging it wastes time. The usual causes are prefix changes — a timestamp, an id, unstable key ordering, a per-user tool list — or, on GPT-6, changing top-level reasoning effort between turns instead of appending a configuration_update item. Too much traffic on one cached prefix can still miss through overflow routing, keyed or not.

The fix

On GPT-5.6 and later: keep keys only if you need per-customer cache accounting, and debug misses with Prompt Cache Diagnostics, which compares reuse against an earlier response and names the reason for a miss. On earlier models, keep a stable key per shared prefix as before. Either way, check prefixes byte for byte before blaming routing.

How to tell which case you're in

Start with the model, because the key's job depends on it: before GPT-5.6 it helps routing, and from GPT-5.6 on it only separates accounting. Then compare cached_tokens on requests that should share a prefix. If hits are near zero on a newer model, diff the opening of two consecutive requests rather than their keys — the culprit is usually a byte that changes between them. The Prompt Caching dashboard, filterable by model and service tier, shows whether a low hit rate is confined to one model or appears everywhere.