More Than 4 Cache Breakpoints in One Request

The mistake

Marking more explicit cache breakpoints in a single request than the API actually honours, usually while trying to give a long, multi-section prompt several independent cacheable anchor points — a system section, a tool-definition section, a long reference-document section, a per-user section — and assuming every marked breakpoint gets written.

Why this happens

Only a capped number of new cache writes are registered per request. In implicit mode, the most recent message itself already consumes one of those write slots, which quietly leaves fewer explicit slots than the raw cap suggests if you're not accounting for it. Mark more breakpoints than the cap allows and the request doesn't error — it just silently honours the newest ones and drops the rest, which is the part that actually causes confusion, because nothing in the response tells you a breakpoint was ignored.

Why it matters

A dropped breakpoint doesn't fail loudly — it shows up, if at all, as a section of a long prompt that never seems to hit its expected cache savings no matter how many times it's reused, while other sections in the same request cache normally. Debugging that by staring at the request is close to impossible without already knowing the cap exists, because every marked breakpoint looks identical in the code that sent it.

The fix

Count your breakpoints against the actual published cap before assuming every one you mark gets written, and remember that implicit-mode caching already spends one slot on the latest message by default. For a prompt genuinely structured around more independent cacheable sections than the cap allows, prioritize the sections with the highest expected reuse for the breakpoints you do get, rather than marking every section and letting the API decide silently which ones survive.

Long prompts assembled from several independently-authored chunks — a shared system section, a retrieved-document block, a per-request user section — are the shape most likely to accumulate more breakpoints than the cap allows without anyone deliberately deciding to exceed it; each chunk's author reasonably marks its own boundary, and nobody's counting the total across the whole assembled prompt.

See also

The prompt-caching reference lists the exact cap alongside the separate, larger limit on how many prior breakpoints get considered when matching a read — a different number, easy to conflate with this one.