Home Docs Claude Developer Platform 27 Oct 2026

One change · api

optimizing-for-cost-and-intelligence changedabout-claude/models/optimizing-for-cost-and-intelligence

Nearest release: v2.1.312, published under an hour before this site recorded the change. Shown because the two are within 24 hours of each other. Nothing here says the release caused the edit.

Recorded here 27 Oct 2026 18:58 UTC

Lines+186added

Lines−186removed

From line 17 where the diff opens

First seen 14 Aug 2026 this site's first read of the page

Recorded edits28to this page, all time

The whole hunk

from line 17, old and new numbered

lines

from line 17

1717* **Free wins** cut spend without touching quality: prompt caching, token hygiene, a prompt audit against the model you are running, [batch processing](https://platform.claude.com/docs/en/build-with-claude/batch-processing) at 50% off for work that can wait up to 24 hours, and [workspace spend limits](https://platform.claude.com/docs/en/api/rate-limits#setting-lower-limits-for-workspaces) as the backstop.

1818* **Tradeoffs** exchange cost for intelligence: model choice, effort, output caps and session budgets, an elapsed-time clock, and multi-model architectures.

1919

20Each lever comes with measured results and the rule for when it pays. In Anthropic's measurements, prompt caching was usually the largest lever. The multi-model levers are narrower; a second model was measured in two shapes, an advisor and an orchestrator.

20Each lever comes with measured results and the rule for when it pays. In Anthropic’s measurements, prompt caching was usually the largest lever. The multi-model levers are narrower; a second model was measured in two shapes, an advisor and an orchestrator.

2121

2222## Start here

2323

from line 28

2828| Any workload, any model | Turn on prompt caching and trim unneeded tokens; both are free | [Cache repeated context](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#cache-repeated-context) · [Trim tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) |

2929| A person waits between turns | Count the gaps between your requests, then pick the cache duration from the table in that section | [Pick the cache duration](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#pick-the-cache-duration) |

3030| Costs are too high; quality is fine | Try lower effort on the model you use now | [Tune effort](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#tune-effort) |

31| You are not on the latest model | Upgrade; in Anthropic's measurements each newer model solved at least as many tasks as the one before it, usually for less per solved task | [Upgrade the model](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#upgrade-the-model) |

31| You are not on the latest model | Upgrade; in Anthropic’s measurements each newer model solved at least as many tasks as the one before it, usually for less per solved task | [Upgrade the model](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#upgrade-the-model) |

3232| You are choosing or switching models | Compare on cost per completed task, not per token | [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) |

3333| High-volume work with outputs you can check | Try Claude Haiku 5.5 first | [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) |

34| Quality isn't good enough | If you lowered effort, restore it; otherwise try the next tier up at its default effort | [Tune effort](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#tune-effort) · [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) |

35| Attempts end with `stop_reason: max_tokens` | Raise `max_tokens` to 64,000: in Anthropic's tests only 2 of about 14,000 turns were longer, and cost per solved task stayed about the same | [Set budgets](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#set-budgets-and-output-caps) |

36| You can check outputs (tests, a verifier) | Run at `low` effort and re-run failures at `high`: in Anthropic's tests, about 40% lower cost than everything at `high`, at the same pass rate | [Re-run failures](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#re-run-failures-at-higher-effort) |

34| Quality isn’t good enough | If you lowered effort, restore it; otherwise try the next tier up at its default effort | [Tune effort](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#tune-effort) · [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) |

35| Attempts end with `stop_reason: max_tokens` | Raise `max_tokens` to 64,000: in Anthropic’s tests only 2 of about 14,000 turns were longer, and cost per solved task stayed about the same | [Set budgets](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#set-budgets-and-output-caps) |

36| You can check outputs (tests, a verifier) | Run at `low` effort and re-run failures at `high`: in Anthropic’s tests, about 40% lower cost than everything at `high`, at the same pass rate | [Re-run failures](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#re-run-failures-at-higher-effort) |

3737| Agent loops with a few very costly runs | Set a Claude Managed Agents session budget and a workspace spend limit | [Set budgets](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#set-budgets-and-output-caps) |

38| You want agent runs to finish sooner | Tell the model time matters and show it the elapsed time: 7% to 47% less time in Anthropic's tests, for a few points of score | [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time) |

38| You want agent runs to finish sooner | Tell the model time matters and show it the elapsed time: 7% to 47% less time in Anthropic’s tests, for a few points of score | [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time) |

3939| A lower-cost model stalls only on hard decisions | Try an advisor. It can help only when your model consults it; first price its model alone at low effort and measure the consult rate | [Advisor strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#advisor-strategy-escalate-hard-decisions) |

4040| The work exceeds one context window | Split the work and hand the parts to lower-cost models | [Orchestrator strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#orchestrator-strategy-delegate-bulk-work) |

4141

from line 67

6767 ![Chart, DeepResearch Bench II: caching cuts Fable 5.1 from $37.94 USD to $6.89 USD per task; Sonnet 5, $3.20 USD to $1.18 USD](https://platform.claude.com/docs/images/cost-intel/caching-repriced.svg)

6868</Frame>

6969

70The cache's default lifetime is 5 minutes and an agent loop's turns are seconds apart, so the discount applies to most tokens on every turn. The caching chart's runs read 84% to 90% of their input tokens from the cache.

70The cache’s default lifetime is 5 minutes and an agent loop’s turns are seconds apart, so the discount applies to most tokens on every turn. The caching chart’s runs read 84% to 90% of their input tokens from the cache.

7171

72Pricing other runs' recorded token counts the same way, each model at its default effort, gives the average token cost per task without caching:

72Pricing other runs’ recorded token counts the same way, each model at its default effort, gives the average token cost per task without caching:

7373

7474| Benchmark | Model | Without caching |

7575| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------------------------- |

from line 83

8383<Note>

8484 With Claude Haiku 5.5 on the Messages API, [compact a long conversation](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#manage-the-context-lifecycle) between tasks, before its prompt passes 100,000 tokens. Past that, every token of the request costs 5 times as much, output included. Cache reads and writes count toward the 100,000 (see [Long context pricing](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing)).

8585

86 If you use [compaction at a token threshold](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold#parameters) instead, set its `trigger` a little below 100,000 minus your prompt's largest growth between requests (the default is 150,000). In Anthropic's test with 10,000-token turns, a trigger of 90,000 cost 70% less per request than the default[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

86 If you use [compaction at a token threshold](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold#parameters) instead, set its `trigger` a little below 100,000 minus your prompt’s largest growth between requests (the default is 150,000). In Anthropic’s test with 10,000-token turns, a trigger of 90,000 cost 70% less per request than the default[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

8787</Note>

8888

8989#### Pick the cache duration

from line 92

9292

9393The 1-hour duration costs more to write (2x the input price instead of 1.25x). A miss on either duration bills the whole prefix at the write price instead of the read price.

9494

95Choose the cache duration when a conversation starts. In Anthropic's test, the first turn added at 1 hour to a conversation cached at 5 minutes wrote the whole prompt again, once: $0.80 USD at 100,000 tokens on Claude Opus 5.5, compared with $0.02 USD of input for a turn that read the cache[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

95Choose the cache duration when a conversation starts. In Anthropic’s test, the first turn added at 1 hour to a conversation cached at 5 minutes wrote the whole prompt again, once: $0.80 USD at 100,000 tokens on Claude Opus 5.5, compared with $0.02 USD of input for a turn that read the cache[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

9696

97Anthropic measured a small agent that triages bug reports ([Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens)) with pauses inserted before some turns to simulate a person's delay[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). Accuracy stayed within run-to-run noise at every setting measured. Anthropic also measured extra requests that keep the 5-minute cache warm (keep-alive requests).

97Anthropic measured a small agent that triages bug reports ([Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens)) with pauses inserted before some turns to simulate a person’s delay[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). Accuracy stayed within run-to-run noise at every setting measured. Anthropic also measured extra requests that keep the 5-minute cache warm (keep-alive requests).

9898

9999To decide, count the gaps between consecutive requests in a conversation, and find your row in the following table:

100100

from line 108

108108| Pauses run for minutes | Claude Fable 5.1 | 5-minute cache kept warm, instead of the 1-hour duration |

109109| Pauses run toward an hour | Claude Fable 5.1 | 1-hour duration |

110110

111On Claude Haiku 5.5, use the "Any" rows: it reads the cache at the standard 0.1x the input price, twice the 0.05x of Claude Opus 5.5 and Claude Sonnet 5.5, so by list prices each keep-alive request costs twice as much relative to the input price. A `max_tokens: 0` request refreshed its cache in a separate test (reference 16).

111On Claude Haiku 5.5, use the “Any” rows: it reads the cache at the standard 0.1x the input price, twice the 0.05x of Claude Opus 5.5 and Claude Sonnet 5.5, so by list prices each keep-alive request costs twice as much relative to the input price. A `max_tokens: 0` request refreshed its cache in a separate test (reference 16).

112112

113113**Why, row by row.**

114114

from line 149

149149To keep the cache warm:

150150

1511511. Take the previous request, set `max_tokens` to 0, and drop `stream` if it was set. Do not change a byte of the prefix.

1522. Re-send the request's headers with its body. Without the original request's `anthropic-beta` header, the beta-gated fields in the replayed body are rejected.

1533. Send it within 4 minutes of the previous request's start, and every 4 minutes after that, until the next turn arrives or the session ends. Count from the request's start, because the [cache's 5-minute lifetime](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#how-prompt-caching-works) runs from there.

1522. Re-send the request’s headers with its body. Without the original request’s `anthropic-beta` header, the beta-gated fields in the replayed body are rejected.

1533. Send it within 4 minutes of the previous request’s start, and every 4 minutes after that, until the next turn arrives or the session ends. Count from the request’s start, because the [cache’s 5-minute lifetime](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#how-prompt-caching-works) runs from there.

154154

155That is the [pre-warming request](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#pre-warming-the-cache): it refreshes the cache's lifetime, returns no output, and costs little more than the cache read. Once the cache has expired, the next request, keep-alive or not, re-writes the prefix at the cache write price[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

155That is the [pre-warming request](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#pre-warming-the-cache): it refreshes the cache’s lifetime, returns no output, and costs little more than the cache read. Once the cache has expired, the next request, keep-alive or not, re-writes the prefix at the cache write price[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

156156

157157A `max_tokens: 0` request is rejected in the following cases:

158158

159159* The request sets `thinking.type: "enabled"` (the default adaptive thinking on Claude Fable 5.1 is fine), structured outputs, or a forced tool choice ([its limitations](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#limitations)). On those workloads, buy the 1-hour duration instead.

160* The request carries the top-level `compaction` parameter, so don't re-send a compaction request from [compaction on demand](https://platform.claude.com/docs/en/build-with-claude/compaction-on-demand) as a keep-alive request.

160* The request carries the top-level `compaction` parameter, so don’t re-send a compaction request from [compaction on demand](https://platform.claude.com/docs/en/build-with-claude/compaction-on-demand) as a keep-alive request.

161161

162162<CodeGroup exclude="shell:CLI, python, typescript, csharp, go, java, php, ruby">

163163 ```bash cURL

from line 190

190190

191191Keep per-request text in the newest user turn, after the last cache breakpoint, and make every cache-invalidating change at a natural break, such as the first request after a compaction. The cache is a byte-exact prefix match over the request in order (tools, then system prompt, then messages), so a change anywhere invalidates everything after it.

192192

193When something must change mid-conversation, send it in the form that keeps the cache, where the model supports it. In Anthropic's test on Claude Fable 5.1, Claude Opus 5.5, and Claude Sonnet 5.5, on a cached conversation of about 100,000 tokens, editing the system prompt cost about 25 to 50 times what the same text cost as a mid-conversation system message. Changing effort at the top level cost 20 to 40 times what it cost per message[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs):

193When something must change mid-conversation, send it in the form that keeps the cache, where the model supports it. In Anthropic’s test on Claude Fable 5.1, Claude Opus 5.5, and Claude Sonnet 5.5, on a cached conversation of about 100,000 tokens, editing the system prompt cost about 25 to 50 times what the same text cost as a mid-conversation system message. Changing effort at the top level cost 20 to 40 times what it cost per message[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs):

194194

195195<Frame>

196196 ![Bar chart: system prompt edit up to $1.26 USD, top-level effort change up to $1.02 USD, either in a message up to $0.026 USD](https://platform.claude.com/docs/images/cost-intel/cache-break-cost.svg)

197197</Frame>

198198

199At list prices, the stakes are highest on Claude Fable 5.1 and Claude Mythos 5.1, where a break re-writes the prefix at 1.25x the input price instead of reading it at 0.025x. On Claude Haiku 5.5 a system prompt edit costs 12.5 times an unchanged request. The edit re-wrote the whole conversation. By default, newer accounts get a 400 error instead, except on Claude Mythos 5.1[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). The top-level effort change, in a conversation that held the model's first reply with its thinking, re-wrote all but the system prompt. These figures are from the Messages API.

199At list prices, the stakes are highest on Claude Fable 5.1 and Claude Mythos 5.1, where a break re-writes the prefix at 1.25x the input price instead of reading it at 0.025x. On Claude Haiku 5.5 a system prompt edit costs 12.5 times an unchanged request. The edit re-wrote the whole conversation. By default, newer accounts get a 400 error instead, except on Claude Mythos 5.1[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). The top-level effort change, in a conversation that held the model’s first reply with its thinking, re-wrote all but the system prompt. These figures are from the Messages API.

200200

201201**What to do.**

202202

203* Put nothing that changes per request, such as a timestamp or a queue position, ahead of the stable prefix: it turns every request into a full cache write. In Anthropic's test, a 25-token status line at the front of the system prompt made each request write the whole conversation; moved to the end of the newest user turn, it left the cache read intact[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

203* Put nothing that changes per request, such as a timestamp or a queue position, ahead of the stable prefix: it turns every request into a full cache write. In Anthropic’s test, a 25-token status line at the front of the system prompt made each request write the whole conversation; moved to the end of the newest user turn, it left the cache read intact[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

204204* The [prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#what-invalidates-the-cache) page lists the changes that invalidate the cache, such as a change to [`effort`](https://platform.claude.com/docs/en/build-with-claude/effort), the thinking configuration, the [`speed`](https://platform.claude.com/docs/en/build-with-claude/fast-mode) setting, the system prompt, or a tool definition. Changing the output format invalidates the cache for that conversation (see [structured outputs](https://platform.claude.com/docs/en/build-with-claude/structured-outputs#prompt-modification-and-token-costs)).

205205* Check which models support each form: [mid-conversation system messages](https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages), and [per-message effort](https://platform.claude.com/docs/en/build-with-claude/effort#changing-effort-mid-conversation), which is in beta.

206206* After a cache-invalidating change, confirm cache reads have not dropped; if they have, [cache diagnostics](https://platform.claude.com/docs/en/build-with-claude/cache-diagnostics) shows where the prefix diverged.

207207

208On an earlier model, Anthropic measured when to make such a change, on the triage agent's long sessions[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). An effort change and an added tool cost more mid-session, or on the request that triggered the compaction, than on the first request after compaction. Accuracy was within run-to-run noise. The sessions with no changes cost more than those changed after compaction for another reason: they ran at the default effort throughout, and two of them compacted a second time, while the others started at low effort:

208On an earlier model, Anthropic measured when to make such a change, on the triage agent’s long sessions[18](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). An effort change and an added tool cost more mid-session, or on the request that triggered the compaction, than on the first request after compaction. Accuracy was within run-to-run noise. The sessions with no changes cost more than those changed after compaction for another reason: they ran at the default effort throughout, and two of them compacted a second time, while the others started at low effort:

209209

210210<Frame>

211211 ![Bar chart, cost per triage session: $0.81 USD no changes, $0.95 USD mid-session, $0.92 USD at compaction, $0.75 USD after](https://platform.claude.com/docs/images/cost-intel/compaction-timing.svg)

from line 243

243243* With tool search, the run cost stayed flat at every catalog size.

244244* Accuracy was in the same range either way, and the model never called a wrong tool, so at this scale unused definitions cost money, not correctness.

245245* [Keep your 3 to 5 most used tools loaded](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool#deferred-tool-loading), so Claude can call them without searching first.

246* On the same earlier model, deferring a public GitHub MCP server's toolset (`default_config: {defer_loading: true}` on the [MCP connector](https://platform.claude.com/docs/en/agents-and-tools/mcp-connector)) cut the run cost at the same accuracy. With no tool search tool in the request, no run then called the server's tools.

246* On the same earlier model, deferring a public GitHub MCP server’s toolset (`default_config: {defer_loading: true}` on the [MCP connector](https://platform.claude.com/docs/en/agents-and-tools/mcp-connector)) cut the run cost at the same accuracy. With no tool search tool in the request, no run then called the server’s tools.

247247

248248#### Keep data files out of the prompt

249249

from line 272

272272* Run-wide, the prune and context editing cache about equally well. If you use context editing, [clear in a few large batches](https://platform.claude.com/docs/en/build-with-claude/context-editing#context-editing-and-prompt-caching).

273273

274274<Note>

275 On Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, rewriting an earlier tool result [invalidates every thinking block after it](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#prefix-check), so use compaction instead. The prune was measured on an earlier model without [preserved thinking](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking); on these four models, [context editing](https://platform.claude.com/docs/en/build-with-claude/context-editing) and [compaction](https://platform.claude.com/docs/en/build-with-claude/compaction) run on the server and don't invalidate thinking blocks. [Send assistant turns back exactly as returned](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#append-assistant-turns-exactly-as-returned): dropping their thinking blocks costs output tokens on every turn, because Claude [re-creates the reasoning](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#drop-block-cost).

275 On Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, rewriting an earlier tool result [invalidates every thinking block after it](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#prefix-check), so use compaction instead. The prune was measured on an earlier model without [preserved thinking](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking); on these four models, [context editing](https://platform.claude.com/docs/en/build-with-claude/context-editing) and [compaction](https://platform.claude.com/docs/en/build-with-claude/compaction) run on the server and don’t invalidate thinking blocks. [Send assistant turns back exactly as returned](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#append-assistant-turns-exactly-as-returned): dropping their thinking blocks costs output tokens on every turn, because Claude [re-creates the reasoning](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#drop-block-cost).

276276</Note>

277277

278The prune, adapted from the triage agent's code:

278The prune, adapted from the triage agent’s code:

279279

280280```python

281281import re

from line 319

319319Audit prompts against the model you run now, and again whenever you change models: it is a free win.

320320

321321* Each model generation responds to prompts differently, so a prompt accumulates text written for a model you no longer use.

322* The usual case is over-specific instruction added to compensate for an older model: "verify twice," "be maximally thorough," a mandatory step-by-step procedure, or a hand-rolled reasoning scratchpad.

322* The usual case is over-specific instruction added to compensate for an older model: “verify twice,” “be maximally thorough,” a mandatory step-by-step procedure, or a hand-rolled reasoning scratchpad.

323323* A newer model follows such instructions to the letter, producing extra tool rounds and extra writing that you pay for with no gain in accuracy.

324324

325The audit is one command. The [Claude API skill](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/claude-api-skill) that ships with Claude Code has a `prompt-audit` command that reads a project's prompts and request code and reports what was written for a different model:

325The audit is one command. The [Claude API skill](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/claude-api-skill) that ships with Claude Code has a `prompt-audit` command that reads a project’s prompts and request code and reports what was written for a different model:

326326

327327Let Claude Code audit your prompts for you

328328

from line 375

375375 ![Bar charts per legacy pattern: over-obeyed instructions cost money; broken settings and contradictory rules cost accuracy](https://platform.claude.com/docs/images/cost-intel/prompt-audit-patterns.svg)

376376</Frame>

377377

378* Instructions the new model follows too literally cost money: removing "verify twice" cut Claude Opus 5's cost per ticket, and removing "be maximally thorough" cut it almost as much.

379* Text that no longer fits the model costs accuracy instead: a retired thinking setting, contradictory rules, and a hand-rolled scratchpad that conflicts with the model's own thinking each restored accuracy on Claude Opus 5 when removed.

378* Instructions the new model follows too literally cost money: removing “verify twice” cut Claude Opus 5’s cost per ticket, and removing “be maximally thorough” cut it almost as much.

379* Text that no longer fits the model costs accuracy instead: a retired thinking setting, contradictory rules, and a hand-rolled scratchpad that conflicts with the model’s own thinking each restored accuracy on Claude Opus 5 when removed.

380380

381381The same patterns tend to appear in tool descriptions and skills, which are worth auditing too.

382382

from line 404

407| GPQA Diamond[9](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | Claude Haiku 5.5 | `medium` (default) | 85% accuracy, compared with 91% for each of Claude Opus 5.5 and Claude Sonnet 5.5 | About a twentieth of Claude Opus 5.5's cost per correct answer and about a tenth of Claude Sonnet 5.5's |

407| GPQA Diamond[9](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | Claude Haiku 5.5 | `medium` (default) | 85% accuracy, compared with 91% for each of Claude Opus 5.5 and Claude Sonnet 5.5 | About a twentieth of Claude Opus 5.5’s cost per correct answer and about a tenth of Claude Sonnet 5.5’s |

409409

410410**What to do.**

from line 411

411411

412412* Start with Claude Opus 5.5 at its default effort: on the SWE-bench Pro subset it matched Claude Fable 5.1 at its default, inside run-to-run noise, for about a fifth of the cost per solved task. Opus 5.5 and Fable 5.1 both largely saturate the SWE-bench Pro subset. Use Claude Fable 5.1 for demanding reasoning and long-horizon agentic work, or when your evals on Claude Opus 5.5 at higher effort still fall short.

413413* At the other end, Claude Haiku 5.5 suits high-volume and latency-sensitive work with checkable outputs.

414* Price every candidate in cost per completed task on your own traffic, including Claude Opus 5.5 at its default effort and the frontier model at reduced effort. Each of Claude Opus 5.5's token prices, cache reads included, is twice Claude Sonnet 5.5's, but on the SWE-bench Pro subset Opus 5.5 at `low` solved about the same share of tasks as Sonnet 5.5 at its default, inside run-to-run noise, for about 7% more per solved task.

414* Price every candidate in cost per completed task on your own traffic, including Claude Opus 5.5 at its default effort and the frontier model at reduced effort. Each of Claude Opus 5.5’s token prices, cache reads included, is twice Claude Sonnet 5.5’s, but on the SWE-bench Pro subset Opus 5.5 at `low` solved about the same share of tasks as Sonnet 5.5 at its default, inside run-to-run noise, for about 7% more per solved task.

415415* The ranking flips by workload, and no price list tells you which way, so measure it on your own workload. On long research loops, Claude Fable 5.1 can do more work, not less, and so cost more per task (DeepResearch Bench II[7](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs)).

416416* Price the tail of your workload, not the median: compare models on the hardest tenth of your tasks, not the typical one.

417417

from line 425

425425

426426### Upgrade the model

427427

428If you are a model or two behind, the cheapest lever is the model string. In Anthropic's measurements, each newer model solved at least as many tasks as the one before it, usually for less per solved task. The chart plots recent Claude models on two benchmarks in which an agent works alone on one written task and tests check the result: the SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) subset, where every model ran through the same test setup, and Terminal-Bench 3[20](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). The table lists what each upgrade changed:

428If you are a model or two behind, the cheapest lever is the model string. In Anthropic’s measurements, each newer model solved at least as many tasks as the one before it, usually for less per solved task. The chart plots recent Claude models on two benchmarks in which an agent works alone on one written task and tests check the result: the SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) subset, where every model ran through the same test setup, and Terminal-Bench 3[20](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). The table lists what each upgrade changed:

429429

430430<Frame>

431431 ![Charts: on SWE-bench Pro, Opus 5.5 and Sonnet 5.5 cut cost per solved task most; on Terminal-Bench 3, Opus drops to $28 USD](https://platform.claude.com/docs/images/cost-intel/upgrade-ladder-sonnet-5-5.svg)

from line 434

435435| ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | --------------------------------------- | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |

440440| DeepResearch Bench II[7](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | Claude Fable 5 to Claude Fable 5.1 | `high` | 2 to 3 more points on the 21 tasks where no attempt was cut short | 46% more per task on all 50 tasks |

from line 455

455455

456456Sweep effort on your own workload before you add a second model. Lower effort costs less and is often faster, and how much accuracy it gives up depends on the task: on long-horizon coding, effort bought accuracy, and on research tasks it bought a lot in one measurement and nothing in another. The [`effort`](https://platform.claude.com/docs/en/build-with-claude/effort) parameter governs how much thinking, tool calling, and self-verification the model does; `high` is the default on most models, and `medium` on Claude Opus 5.5 and Claude Haiku 5.5.

457457

458The following table lists Anthropic's measurements on two benchmarks with current models; scores and costs are not comparable between the two benchmarks. Costs are per task, not per solved task as in [Compare models on cost per task](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task). Each score pools two runs on SWE-bench Pro and three on DeepResearch Bench II, unless its cell says otherwise:

458The following table lists Anthropic’s measurements on two benchmarks with current models; scores and costs are not comparable between the two benchmarks. Costs are per task, not per solved task as in [Compare models on cost per task](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task). Each score pools two runs on SWE-bench Pro and three on DeepResearch Bench II, unless its cell says otherwise:

459459

461461| --------------------------------------------------------------------------------------------------------------------------- | ---------------- | ------------------ | ------------------------------------ | ------------- |

from line 482

482482

483483* Sweep two or three effort levels on a sample of your own traffic, each in a separate session: changing top-level effort mid-session [invalidates the cache](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#cache-repeated-context) and distorts the comparison. The curve is the baseline any multi-model strategy must beat ([step 2 of measuring on your own workload](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#measure-on-your-own-workload)).

484484* On long-horizon coding, effort is a real tradeoff, which [re-running failures at higher effort](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#re-run-failures-at-higher-effort) turns back into a saving. At `medium`, Claude Sonnet 5.5 scored about 7 points lower than at its default, `high`, on the SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) subset for about 59% of the cost per task (2 runs each), and scored 62.7% compared with 77.8% on the internal agentic-coding benchmark[11](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) (1 attempt per task).

485* At `xhigh` and `max`, Claude Sonnet 5.5 can start its own rounds of review after it finishes a task, which takes more time and tokens. To keep that work on the task, add the prompt under "Thoroughness at `xhigh` and `max` effort" in [Prompting Claude Sonnet 5.5](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-sonnet-5-5#steer-initiative-and-scope) to your system prompt.

485* At `xhigh` and `max`, Claude Sonnet 5.5 can start its own rounds of review after it finishes a task, which takes more time and tokens. To keep that work on the task, add the prompt under “Thoroughness at `xhigh` and `max` effort” in [Prompting Claude Sonnet 5.5](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-sonnet-5-5#steer-initiative-and-scope) to your system prompt.

486486* On research tasks, measure the model you ship. Claude Fable 5.1 scored nearly the same from `low` to `high` on DeepResearch Bench II; Claude Opus 5.5 scored 13.3 points lower at `low` than at `medium` on DRACO[21](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs). The two results come from different models on different benchmarks, so they show that the answer varies, not why.

487* At `low`, Claude Opus 5.5's typical task took 84% less time than at `medium` on DRACO[21](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), 30% less on HLE[22](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), and 47% less on an internal set of physics problems[23](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), and it scored 13.3, 3.4, and 14.0 points lower (3, 3, and 4 attempts per task).

487* At `low`, Claude Opus 5.5’s typical task took 84% less time than at `medium` on DRACO[21](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), 30% less on HLE[22](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), and 47% less on an internal set of physics problems[23](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), and it scored 13.3, 3.4, and 14.0 points lower (3, 3, and 4 attempts per task).

488488* On an internal defect-search benchmark for work larger than one context window[8](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), Claude Fable 5.1 took 15.2, 17.5, and 19.9 hours per run at `low`, `medium`, and `high` (3 runs each). To save time, also compare lower effort with the two changes in [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time): `low` effort saved more time, and the changes kept more of the score.

489489

490490### Re-run failures at higher effort

491491

492When a task's outcome is checkable, the cheapest policy on the effort curve is not a fixed setting: run every task at a low setting and re-run only the failures at a higher one.

492When a task’s outcome is checkable, the cheapest policy on the effort curve is not a fixed setting: run every task at a low setting and re-run only the failures at a higher one.

493493

494494Anthropic computed this policy task by task from the Claude Opus 5.5 effort runs on the SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) subset in [Tune effort](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#tune-effort):

495495

from line 508

508508* At `low`, 13% of tasks failed. With those re-run at `high`, the pass rate was slightly higher than with everything at `high`, for a little over half the cost.

509509* Use this policy for the saving, not the lift. Most of the small lift is the second attempt: re-running the failures of one `high` run at `high` scores about the same, for more money.

510510

511Two conditions apply. First, you need a failure signal (here, the benchmark's own tests); a checker that passes bad work lets those failures through. Second, every first-pass failure takes two runs' worth of wall-clock time, so the saving is paid for in latency on the failures.

511Two conditions apply. First, you need a failure signal (here, the benchmark’s own tests); a checker that passes bad work lets those failures through. Second, every first-pass failure takes two runs’ worth of wall-clock time, so the saving is paid for in latency on the failures.

512512

513513### Set budgets and output caps

514514

515515Two controls do two different jobs: a session budget caps what one Claude Managed Agents session can spend, and `max_tokens` caps the length of one response. Set each where it applies, with a [workspace spend limit](https://platform.claude.com/docs/en/api/rate-limits#setting-lower-limits-for-workspaces) as the final backstop.

516516

517* **Session budgets on Claude Managed Agents** are a hard stop on spend. A [session budget](https://platform.claude.com/docs/en/managed-agents/budgets) is a dollar cap on one session's [list cost](https://platform.claude.com/docs/en/managed-agents/budgets#how-list-cost-is-measured), which the platform meters for tokens, web searches, and session running time.

517* **Session budgets on Claude Managed Agents** are a hard stop on spend. A [session budget](https://platform.claude.com/docs/en/managed-agents/budgets) is a dollar cap on one session’s [list cost](https://platform.claude.com/docs/en/managed-agents/budgets#how-list-cost-is-measured), which the platform meters for tokens, web searches, and session running time.

518518* At the cap, or a fraction past it, the session pauses with `stop_reason: budget_reached`; raising the budget resumes it. It is platform-enforced and works on any model with a list price. Deployments apply the same budget to each session they start.

519519* **`max_tokens`** caps a single response, invisibly to the model, so lowering it does not make the model economize. The turns that needed the room are discarded and still billed. Retrying capped attempts rarely helps: at the same cap most of them fail again, and at a higher one you also pay for the wasted attempt.

520520* Set `max_tokens` to 64,000 for agentic work, or to 128,000, the maximum, when a single cut-off attempt is costly. [Stream responses](https://platform.claude.com/docs/en/build-with-claude/streaming) that large, treat [`stop_reason: max_tokens`](https://platform.claude.com/docs/en/build-with-claude/handling-stop-reasons#max-tokens) as a failure, and save money with effort, which the model can see.

from line 559

559559

560560### Show the model elapsed time

561561

562To make an agent finish sooner and cost less, tell it that time matters, and show it the elapsed time on every request. In Anthropic's measurement, the two changes cut the time of a single Claude Opus 5.5 agent at `medium` effort by 47% and its cost by 60% on the typical (median) DRACO[21](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) research task, and its average score was 4.1 points lower.

562To make an agent finish sooner and cost less, tell it that time matters, and show it the elapsed time on every request. In Anthropic’s measurement, the two changes cut the time of a single Claude Opus 5.5 agent at `medium` effort by 47% and its cost by 60% on the typical (median) DRACO[21](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) research task, and its average score was 4.1 points lower.

563563

564564On HLE[22](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) and an internal physics set[23](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs), the typical task took 14% and 7% less time, and the physics score was 3.0 points lower. Lowering effort to `low` saved more time and cost than the changes did, and lost more score (reference 21). Check the score on your own tasks before you adopt the changes.

565565

from line 575

575575

576576## Combine models

577577

578Tune one model before you add a second: sweep effort on your current model first, because most workloads end there, and if the sweep shows a gap, price the stronger model alone at low effort. Build a pairing only if it beats the single model's whole curve of score against spend. The multi-model results on this page were judged against the same model at lower effort and against the next model down running alone, which is the comparison to run on your own workload.

578Tune one model before you add a second: sweep effort on your current model first, because most workloads end there, and if the sweep shows a gap, price the stronger model alone at low effort. Build a pairing only if it beats the single model’s whole curve of score against spend. The multi-model results on this page were judged against the same model at lower effort and against the next model down running alone, which is the comparison to run on your own workload.

579579

580580The following table compares the two strategies:

581581

583583| ---------------- | ----------------------------------------------------- | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------------------------- |

585585| **Orchestrator** | Frontier model runs the loop, delegates the bulk work | Plans, dispatches, and synthesizes | Work that fans out across genuinely independent files, documents, or cases, especially more than one context window of it | How hard the pieces are to coordinate |

586586

587587### Advisor strategy: escalate hard decisions

from line 588

588588

589589An advisor can help only when the executor consults it.

590590

591In Anthropic's measurements on current models, the one pairing whose executor consulted often, on a coding benchmark, scored 1.7 points higher than the executor alone at the same effort, at the edge of run-to-run noise, for about 2.1 times the cost. Where the executor rarely or never consulted, the pairing scored within run-to-run noise of the executor alone, or lower.

591In Anthropic’s measurements on current models, the one pairing whose executor consulted often, on a coding benchmark, scored 1.7 points higher than the executor alone at the same effort, at the edge of run-to-run noise, for about 2.1 times the cost. Where the executor rarely or never consulted, the pairing scored within run-to-run noise of the executor alone, or lower.

592592

593593To turn it on, add the [advisor tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool), a beta feature, to your request. On Claude Managed Agents, [give the session an advisor](https://platform.claude.com/docs/en/managed-agents/multiagent-orchestration#give-the-session-an-advisor). For Claude Code, see [escalating hard decisions with the advisor tool](https://code.claude.com/docs/en/advisor).

594594

from line 605

605605| Chartography[13](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | 61.7, 7 points below the executor alone and beyond run-to-run noise | About the same as the executor alone | On 1 of 300 tasks |

606606

607607* **Internal agentic-coding benchmark.** Claude Opus 5.5 at `high`, with a Claude Fable 5.1 advisor.

608* **GPQA Diamond.** Claude Haiku 5.5 or Claude Sonnet 5.5 at each model's default (`medium` and `high`), with a Claude Opus 5.5 advisor.

608* **GPQA Diamond.** Claude Haiku 5.5 or Claude Sonnet 5.5 at each model’s default (`medium` and `high`), with a Claude Opus 5.5 advisor.

609609* **Chartography.** Claude Opus 5.5 at `low`, with a Claude Fable 5.1 advisor.

610610

611The advisor can only hand over capability the executor lacks: on GPQA Diamond, Claude Sonnet 5.5 alone tied Claude Opus 5.5 alone at 91% with refusals counted as wrong, but only because Opus 5.5 refused six biology questions per run compared with Sonnet 5.5's one. On the questions each answered, Opus 5.5 led by about 2 points.

611The advisor can only hand over capability the executor lacks: on GPQA Diamond, Claude Sonnet 5.5 alone tied Claude Opus 5.5 alone at 91% with refusals counted as wrong, but only because Opus 5.5 refused six biology questions per run compared with Sonnet 5.5’s one. On the questions each answered, Opus 5.5 led by about 2 points.

612612

613On the coding benchmark, the pairing's gain is at the edge of run-to-run noise, and it lands about on Claude Opus 5.5's own effort curve, so the advisor bought about what more effort does. The following chart plots the pairing against that curve and Claude Fable 5.1's from August:

613On the coding benchmark, the pairing’s gain is at the edge of run-to-run noise, and it lands about on Claude Opus 5.5’s own effort curve, so the advisor bought about what more effort does. The following chart plots the pairing against that curve and Claude Fable 5.1’s from August:

614614

615615<Frame>

616616 ![Coding benchmark: Opus 5.5 at high with a Fable 5.1 advisor gains 1.7 points for 2.1 times the cost, near its effort curve](https://platform.claude.com/docs/images/cost-intel/internal-coding-advisor-opus-5-5.svg)

from line 618

618618

619619**What to do.**

620620

621* Measure the consult rate on your own tasks. The advisor tool's definition, about 1,000 prompt tokens per request, adds cost even when the executor never consults. With [prompt caching](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#cache-repeated-context), a multi-turn agent loop pays for the definition at the cache-read price after the first request, so this overhead matters most for short, single-request tasks.

621* Measure the consult rate on your own tasks. The advisor tool’s definition, about 1,000 prompt tokens per request, adds cost even when the executor never consults. With [prompt caching](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#cache-repeated-context), a multi-turn agent loop pays for the definition at the cache-read price after the first request, so this overhead matters most for short, single-request tasks.

622622* If the executor rarely consults, see [Prompting for coding and agent tasks](https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool#prompting-for-coding-and-agent-tasks) for system prompts that ask for consultations, and measure again after you add one. On 20 SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) problems, the suggested prompt made no clear difference for Claude Sonnet 5.5.

623* Restore the executor's effort if the consult rate collapses. In measurements on earlier models, one pairing that consulted on most tasks at the default effort consulted on few when effort was lowered, and then scored below the executor alone at its default effort (SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs)).

623* Restore the executor’s effort if the consult rate collapses. In measurements on earlier models, one pairing that consulted on most tasks at the default effort consulted on few when effort was lowered, and then scored below the executor alone at its default effort (SWE-bench Pro[3](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs)).

624624* Recheck at every model release, because releases move both the capability gap and the price ratio.

625625

626626### Orchestrator strategy: delegate bulk work

from line 630

630630* Routine work with a long cost tail (an easy slice of BrowseComp[4](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs))

631631* Work larger than one context window (an internal defect-search benchmark[8](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs))

632632

633When the work is one dependent chain, or fits in a single context without a long cost tail, the orchestrator's model alone at lower effort came out ahead in every case measured, also on earlier models (the harder BrowseComp set and DeepWideSearch[6](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs)); independent external work reports the same pattern[5](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

633When the work is one dependent chain, or fits in a single context without a long cost tail, the orchestrator’s model alone at lower effort came out ahead in every case measured, also on earlier models (the harder BrowseComp set and DeepWideSearch[6](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs)); independent external work reports the same pattern[5](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs).

634634

635635To build an orchestrator, use [multiagent orchestration](https://platform.claude.com/docs/en/managed-agents/multiagent-orchestration) in Claude Managed Agents: configure an orchestrator agent and worker agents that it can call, each with its own model.

636636

from line 640

640640

641641The [Model selection matrix](https://platform.claude.com/docs/en/about-claude/models/choosing-a-model#model-selection-matrix) lists Claude Haiku 5.5 for sub-agent tasks; before you use it as a worker, [measure it on your own workload](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#measure-on-your-own-workload).

642642

643The following chart plots the defect-search benchmark's configurations by cost and score:

643The following chart plots the defect-search benchmark’s configurations by cost and score:

644644

645645<Frame>

646646 ![Chart, defect-search benchmark: the orchestrator costs about half of Fable 5.1 alone at any effort and scores 10 to 12 points below](https://platform.claude.com/docs/images/cost-intel/corpus-pareto.svg)

from line 656

656656Measure on your own workload: the numbers on this page reflect list prices at the time of measurement and will drift as models and prices change, and your escalation rate, how cleanly tasks split, and transcript length move them too. The method stays the same:

657657

6586581. Pull a few tasks from production logs, weighted like real traffic, and [write outcome checks](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests) for each: tests pass, ticket closed, row count correct. Record cost per task beside the score.

6592. Baseline the model tiers across effort levels, not only the default, and plot score against spend. A multi-model configuration must beat the single model's whole curve.

6603. If the curve shows a gap effort can't close, add the multi-model strategy that fits and re-run the suite.

6592. Baseline the model tiers across effort levels, not only the default, and plot score against spend. A multi-model configuration must beat the single model’s whole curve.

6603. If the curve shows a gap effort can’t close, add the multi-model strategy that fits and re-run the suite.

6616614. Run the winner in shadow on a traffic slice before cutover, then keep the suite running.

662662

663**How to price a task.** Price each of the five token counts in a response's `usage` at its own rate: uncached input, 5-minute and 1-hour cache writes at 1.25x and 2x the input price, cache reads, and output. Sum them across the task's requests (the [Usage and Cost API](https://platform.claude.com/docs/en/manage-claude/usage-cost-api) reports the aggregate). On Claude Haiku 5.5, use the rates for each request's prompt length, counting cache reads and writes (see [Long context pricing](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing)). When `usage.speed` is `"fast"`, use the [fast mode rates](https://platform.claude.com/docs/en/about-claude/pricing#fast-mode-pricing).

663**How to price a task.** Price each of the five token counts in a response’s `usage` at its own rate: uncached input, 5-minute and 1-hour cache writes at 1.25x and 2x the input price, cache reads, and output. Sum them across the task’s requests (the [Usage and Cost API](https://platform.claude.com/docs/en/manage-claude/usage-cost-api) reports the aggregate). On Claude Haiku 5.5, use the rates for each request’s prompt length, counting cache reads and writes (see [Long context pricing](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing)). When `usage.speed` is `"fast"`, use the [fast mode rates](https://platform.claude.com/docs/en/about-claude/pricing#fast-mode-pricing).

664664

665The following example computes one request's cost at Claude Opus 5.5's list prices. Change the three prices for another model:

665The following example computes one request’s cost at Claude Opus 5.5’s list prices. Change the three prices for another model:

666666

667667<CodeGroup exclude="shell:cURL, shell:CLI">

668668 <CodeGroupItem language="cURL">

from line 852

852852**What to check.**

853853

854854* **Cache-read share.** In agent loops most input tokens should be cache reads. If `cache_read_input_tokens` is small next to `input_tokens` plus `cache_creation_input_tokens`, check that caching is engaged and that the prefix stays the same between requests.

855* **Advisor tool and compaction.** When the [advisor tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/advisor-tool#usage-and-billing) or [compaction](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold#understanding-usage) is enabled, some tokens are reported only in `usage.iterations` and not in the top-level totals. Sum over `usage.iterations` instead, and price `advisor_message` entries at the advisor model's rates.

856856

857The following table lists the levers in the order to try them. The figures come from Anthropic's tests, launch posts, and published prices and speeds:

857The following table lists the levers in the order to try them. The figures come from Anthropic’s tests, launch posts, and published prices and speeds:

858858

860860| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

861861| Prompt caching | Cost cut by a factor of 2.7 to 5.5 on a research agent, and by 83% to 86% on a small triage agent | None | Faster | [Cache repeated context](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#cache-repeated-context) |

862862| 1-hour cache duration | With 1 turn in 10 after a pause, the 5-minute default cost 54% more; with 1 in 20, about the same | None | On earlier models, stayed warm after a pause[16](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | [Pick the cache duration](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#pick-the-cache-duration) |

863| Input trimming | A further 21% to 26% off a small triage agent's bill | With tool search, accuracy in the same range as without[27](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | Neutral | [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) |

864| Tool search | 45% with 500 tool definitions attached; 20% from deferring a GitHub MCP server's toolset | None for tools the task never needs | Neutral | [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) |

863| Input trimming | A further 21% to 26% off a small triage agent’s bill | With tool search, accuracy in the same range as without[27](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#refs) | Neutral | [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) |

864| Tool search | 45% with 500 tool definitions attached; 20% from deferring a GitHub MCP server’s toolset | None for tools the task never needs | Neutral | [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) |

865865| Data files through code execution | 92% on a data task of 25 questions, with prompt caching off | A gain: 25 of 25 correct instead of 7 | Faster | [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) |

867867| Prompt audit against the current model | 14% on both model upgrades tested | None; a gain on one | Faster (fewer tool rounds) | [Audit prompts against the current model](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#audit-prompts-against-the-current-model) |

from line 871

872872| Raising `max_tokens` | None: cost per solved task stayed about the same | A gain: up to 22 points more tasks solved on one coding benchmark; none on another | Neutral | [Set budgets and output caps](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#set-budgets-and-output-caps) |

873873| Advisor | None: about 2.1 times the cost of the model alone, on a coding benchmark | On coding, a small gain that may be chance; on chart reading, a loss | About 1.4 extra calls per attempt, on coding | [Advisor strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#advisor-strategy-escalate-hard-decisions) |

874| Orchestrator | About half the cost of the orchestrator's model alone, on work larger than one context window; on routine work, measure first | 10 to 12 points lower on that work | Much faster on that work | [Orchestrator strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#orchestrator-strategy-delegate-bulk-work) |

874| Orchestrator | About half the cost of the orchestrator’s model alone, on work larger than one context window; on routine work, measure first | 10 to 12 points lower on that work | Much faster on that work | [Orchestrator strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#orchestrator-strategy-delegate-bulk-work) |

875875| Fast mode | None: twice the standard token prices | None | Up to 2.5x higher output tokens per second | [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time) |

876876

877877### Automate the search with the Claude API skill

878878

879Run the [Claude API skill](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/claude-api-skill)'s `cost-optimize` command before you search for savings. It profiles where your tokens go and applies the levers in order, caching first:

879Run the [Claude API skill](https://platform.claude.com/docs/en/agents-and-tools/agent-skills/claude-api-skill)’s `cost-optimize` command before you search for savings. It profiles where your tokens go and applies the levers in order, caching first:

880880

881881Let Claude Code find savings for you

882882

from line 902

902902 ![Scatter chart, tau2-bench retail: each search step from Claude Opus 5; Sonnet 5 at low is 61% cheaper within 2 points](https://platform.claude.com/docs/images/cost-intel/hillclimb.svg)

903903</Frame>

904904

905Two settings determine whether you can trust the search's answer:

905Two settings determine whether you can trust the search’s answer:

906906

907907* **Hold out a split the search never reads.** The search judges adoption there, so confirmation catches a change that fits only the tasks the search saw, as it did the prompt edits (reference 24).

908* **Repeat the baseline before the first change.** Identical runs of one starting configuration differed by several tasks in these measurements (reference 24). The repeats give the search's adoption gate a noise bar.

909909

910910## Benchmarks referenced

911911

912Except where a reference says otherwise, measurements are Anthropic-internal runs of these benchmarks. Unless noted, costs are USD at the list prices in effect when each benchmark ran; Claude Sonnet 5 figures use $2 USD and $10 USD per million input and output tokens. Claude Sonnet 5.5 figures from the September 24, 2026, runs use the same input and output prices and its current cache-read price, $0.10 USD per million tokens, rather than its launch price of $0.20 USD, applied to each run's recorded token counts. Those runs used a pre-release build of Claude Sonnet 5.5, and no attempt in them was answered by a fallback model. Charts labeled "notional USD" price each request's token counts at those rates rather than reporting invoices.

912Except where a reference says otherwise, measurements are Anthropic-internal runs of these benchmarks. Unless noted, costs are USD at the list prices in effect when each benchmark ran; Claude Sonnet 5 figures use $2 USD and $10 USD per million input and output tokens. Claude Sonnet 5.5 figures from the September 24, 2026, runs use the same input and output prices and its current cache-read price, $0.10 USD per million tokens, rather than its launch price of $0.20 USD, applied to each run’s recorded token counts. Those runs used a pre-release build of Claude Sonnet 5.5, and no attempt in them was answered by a fallback model. Charts labeled “notional USD” price each request’s token counts at those rates rather than reporting invoices.

913913

9141. **WideSearch:** Wong et al., "WideSearch: Benchmarking Agentic Broad Info-Seeking," arXiv:2508.07999, 2025. Broad web-research tasks graded on a many-row table's completeness and accuracy; 200 problems, 3 runs per configuration, run August 1 to 2, 2026. With Claude Fable 5 on this benchmark, DeepWideSearch (reference 6), BrowseComp (reference 4), and GDPval (reference 2), `low` effort gave up 1 to 3 points for a third to a half off the cost per task, `medium` matched the default's accuracy at about 70% to 87% of its cost, and the default bought nothing measurable over `medium` on any of the four. The cost-concentration chart is a separate 20-problem run, 3 runs per problem, run August 3 to 4, 2026, costed from per-request billing records. It covers 59 of the 60 runs. The sixtieth, billed $10.91 USD of the $432.31 USD total, has no cost recorded with its result and is not included.

9141. **WideSearch:** Wong et al., “WideSearch: Benchmarking Agentic Broad Info-Seeking,” arXiv:2508.07999, 2025. Broad web-research tasks graded on a many-row table’s completeness and accuracy; 200 problems, 3 runs per configuration, run August 1 to 2, 2026. With Claude Fable 5 on this benchmark, DeepWideSearch (reference 6), BrowseComp (reference 4), and GDPval (reference 2), `low` effort gave up 1 to 3 points for a third to a half off the cost per task, `medium` matched the default’s accuracy at about 70% to 87% of its cost, and the default bought nothing measurable over `medium` on any of the four. The cost-concentration chart is a separate 20-problem run, 3 runs per problem, run August 3 to 4, 2026, costed from per-request billing records. It covers 59 of the 60 runs. The sixtieth, billed $10.91 USD of the $432.31 USD total, has no cost recorded with its result and is not included.

915915

9162. **GDPval:** OpenAI, "GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks," 2025. Knowledge-work deliverables graded against task rubrics; a 210-task run of the released gold set, one attempt per task, run August 2, 2026. A Claude model grades, so absolute scores may differ from published results.

9162. **GDPval:** OpenAI, “GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks,” 2025. Knowledge-work deliverables graded against task rubrics; a 210-task run of the released gold set, one attempt per task, run August 2, 2026. A Claude model grades, so absolute scores may differ from published results.

917917

9183. **SWE-bench Pro:** Scale AI, "SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?", 2025.

9183. **SWE-bench Pro:** Scale AI, “SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?”, 2025.

919919

920 * **What was measured:** A 482-problem subset selected for compatibility with Anthropic's evaluation harness; scores are not comparable to the public leaderboard. They are also not comparable to the SWE-bench Pro results in the Claude Opus 5.5 system card, which come from runs at `max` effort on a different problem set.

921 * **How priced:** Costs on this subset are priced as a customer's organization is metered: each request's prior prompt as a cache read and its new tokens as a 5-minute cache write, from the runs' own usage records, checked against a customer ledger.

920 * **What was measured:** A 482-problem subset selected for compatibility with Anthropic’s evaluation harness; scores are not comparable to the public leaderboard. They are also not comparable to the SWE-bench Pro results in the Claude Opus 5.5 system card, which come from runs at `max` effort on a different problem set.

921 * **How priced:** Costs on this subset are priced as a customer’s organization is metered: each request’s prior prompt as a cache read and its new tokens as a 5-minute cache write, from the runs’ own usage records, checked against a customer ledger.

922922 * **Claude Opus 5.5 runs:** Claude Opus 5.5 figures average two runs at `low`, `medium` (its default), and `high`, and use one run at `xhigh`, all run September 19 to 20, 2026, with the same 16,384-token cap per turn as the August Claude Opus 5 runs; the cap cut off 2 `xhigh` attempts and none at other settings. Measured against `high`, Claude Opus 5.5 scored about 2.5 points lower at its default, `medium`, for about 70% of the cost per task, and about 8 points lower at `low` for about a third of the cost; `xhigh` scored about 1.4 points higher for 2.5 times the cost of `high`. The ratios come from unrounded costs.

923923 * **The 478 problems:** The Opus 5.5 runs used a version of the benchmark whose grading containers can reach only internal package mirrors. That version drops one problem whose test needs a live website, and on three more the reference solution fails there, so Opus 5.5 comparisons, and the Claude Fable 5.1 figures set beside them, use the remaining 478 problems.

924924 * **Claude Sonnet 5.5 runs:** Claude Sonnet 5.5 ran the same version of the benchmark: two runs at its default, `high`, and two at `medium`, all on September 24, 2026, with the same 16,384-token cap per turn, which cut off none of its attempts. Its figures use the 478 problems, and so do the Claude Sonnet 5 figures set beside them, counted over those problems from their August runs.

925925 * **Claude Opus 5 runs:** The Claude Opus 5 SWE-bench Pro figures in [Upgrade the model](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#upgrade-the-model) and in this reference average two runs at its default effort and use one run at `low`, all run August 4, 2026.

926926 * **Advisor, Claude Sonnet 5 pairings:** The Claude Sonnet 5 executor pairings with a Claude Opus 5 advisor behind the consult-rate examples in [Advisor strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#advisor-strategy-escalate-hard-decisions) ran on this subset through the same test setup: the pairing at the default effort was run twice (August 7 and August 8, 2026, a run and an exact replication), the low-effort pairing once (August 8, 2026), and, ten days later, Claude Sonnet 5 alone twice at its default effort (77.4%, the baseline for both pairings).

927 * **Advisor, Claude Sonnet 5.5 consult check:** The Claude Sonnet 5.5 consult check cited in [Advisor strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#advisor-strategy-escalate-hard-decisions) ran one attempt on each of 20 of the 478 problems, on September 24, 2026, with a Claude Opus 5.5 advisor through the advisor tool. Four configurations ran: `high` and `xhigh`, each with and without the advisor tool documentation's suggested system prompt for coding tasks. At its default effort, `high`, Claude Sonnet 5.5 did not consult the advisor on any of the 20 problems, with or without the prompt. At `xhigh`, it consulted on 3 of the 20 problems without the prompt and 4 with it, each time on its attempts with the most turns.

927 * **Advisor, Claude Sonnet 5.5 consult check:** The Claude Sonnet 5.5 consult check cited in [Advisor strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#advisor-strategy-escalate-hard-decisions) ran one attempt on each of 20 of the 478 problems, on September 24, 2026, with a Claude Opus 5.5 advisor through the advisor tool. Four configurations ran: `high` and `xhigh`, each with and without the advisor tool documentation’s suggested system prompt for coding tasks. At its default effort, `high`, Claude Sonnet 5.5 did not consult the advisor on any of the 20 problems, with or without the prompt. At `xhigh`, it consulted on 3 of the 20 problems without the prompt and 4 with it, each time on its attempts with the most turns.

928928 * **Escalation figures:** Escalation figures come task by task from the Opus 5.5 runs: `low` first, then `high` on its failures, solved 96.4% to 97.5% across run pairings for about $0.17 USD; `medium` first, 96.0% to 97.1% for about $0.24 USD; `high` re-run on its own failures, 96.9% for $0.31 USD; everything at `high`, 94.8% to 95.8% for $0.29 USD.

929 * **Compare models:** The Claude Fable 5.1 figure in [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) comes from one run on the same subset at the default effort, run August 26, 2026: counted over the 478 problems, it solved 92.3% for $1.19 USD per solved task. That section's chart and its Claude Opus 5.5 and Claude Sonnet 5.5 figures pool the two runs at each setting plotted, all counted over the 478 problems: Claude Opus 5.5 at `low`, `medium`, and `high`, and Claude Sonnet 5.5 at `medium` and `high`.

930 * **Compare models, work per task:** In those figures, Claude Opus 5.5 at `low` did less work on each task than Claude Sonnet 5.5 at its default: a little over half of Sonnet 5.5's output tokens and a little over a third of its cache reads. At their defaults, Opus 5.5 at `medium` solved about 5 percentage points more than Sonnet 5.5 for about twice the cost. At that setting it did slightly more work per task than Sonnet 5.5.

931 * **Upgrade the model, runs:** The Claude Fable 5 point in Upgrade the model pools three runs at the default effort, run August 26, 2026, priced as a customer's organization is metered. The upgrade ladder counts every model over the 478 problems, each at its shipped defaults. It uses one run per model except Opus 5, Opus 5.5, Sonnet 5, and Sonnet 5.5 (two runs each) and Fable 5 (the three runs described above). The Opus 4.7, Opus 4.8, Sonnet 4.6, and Sonnet 5 runs ran August 17 to 19, 2026, and the Opus 5 runs on August 4, all in one harness and organization; the Opus 5.5 and Sonnet 5.5 runs are the ones described earlier in this reference.

932 * **Upgrade the model, cost per solved task:** Claude Opus 5 costs 21% more per solved task than Claude Opus 4.8 when those Claude Opus 5 runs are compared with one Claude Opus 4.8 run at its shipped defaults from the August 17 to 19 series, counted over the same 478 problems; the chart's SWE-bench Pro panel shows neither Claude Opus 4.7 nor Claude Opus 4.8. In the same measurements, priced as a customer is billed, Claude Opus 5 solves 12 more points of tasks than Claude Opus 4.8 at that 21% more per solved task, Claude Opus 4.8 solves the same share of tasks as Claude Opus 4.7 for 14% less per solved task, and Claude Opus 5 at `low` effort beats Claude Opus 4.8's default on this benchmark for about 30% of its cost per solved task.

933 * **Upgrade the model, prices:** Claude Sonnet 5's saving over Claude Sonnet 4.6 in Upgrade the model comes from its lower per-token price, which more than offsets the extra tokens it uses per task. Claude Sonnet 5.5 charges the same as Claude Sonnet 5 for everything but cache reads, which cost half as much. Claude Opus 5.5's token prices are lower than Claude Opus 5's, most of all for cache reads. Like Claude Sonnet 5, the frontier tier gained through price: most of Claude Fable 5.1's saving is the lower cache-read price.

929 * **Compare models:** The Claude Fable 5.1 figure in [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) comes from one run on the same subset at the default effort, run August 26, 2026: counted over the 478 problems, it solved 92.3% for $1.19 USD per solved task. That section’s chart and its Claude Opus 5.5 and Claude Sonnet 5.5 figures pool the two runs at each setting plotted, all counted over the 478 problems: Claude Opus 5.5 at `low`, `medium`, and `high`, and Claude Sonnet 5.5 at `medium` and `high`.

930 * **Compare models, work per task:** In those figures, Claude Opus 5.5 at `low` did less work on each task than Claude Sonnet 5.5 at its default: a little over half of Sonnet 5.5’s output tokens and a little over a third of its cache reads. At their defaults, Opus 5.5 at `medium` solved about 5 percentage points more than Sonnet 5.5 for about twice the cost. At that setting it did slightly more work per task than Sonnet 5.5.

931 * **Upgrade the model, runs:** The Claude Fable 5 point in Upgrade the model pools three runs at the default effort, run August 26, 2026, priced as a customer’s organization is metered. The upgrade ladder counts every model over the 478 problems, each at its shipped defaults. It uses one run per model except Opus 5, Opus 5.5, Sonnet 5, and Sonnet 5.5 (two runs each) and Fable 5 (the three runs described above). The Opus 4.7, Opus 4.8, Sonnet 4.6, and Sonnet 5 runs ran August 17 to 19, 2026, and the Opus 5 runs on August 4, all in one harness and organization; the Opus 5.5 and Sonnet 5.5 runs are the ones described earlier in this reference.

932 * **Upgrade the model, cost per solved task:** Claude Opus 5 costs 21% more per solved task than Claude Opus 4.8 when those Claude Opus 5 runs are compared with one Claude Opus 4.8 run at its shipped defaults from the August 17 to 19 series, counted over the same 478 problems; the chart’s SWE-bench Pro panel shows neither Claude Opus 4.7 nor Claude Opus 4.8. In the same measurements, priced as a customer is billed, Claude Opus 5 solves 12 more points of tasks than Claude Opus 4.8 at that 21% more per solved task, Claude Opus 4.8 solves the same share of tasks as Claude Opus 4.7 for 14% less per solved task, and Claude Opus 5 at `low` effort beats Claude Opus 4.8’s default on this benchmark for about 30% of its cost per solved task.

933 * **Upgrade the model, prices:** Claude Sonnet 5’s saving over Claude Sonnet 4.6 in Upgrade the model comes from its lower per-token price, which more than offsets the extra tokens it uses per task. Claude Sonnet 5.5 charges the same as Claude Sonnet 5 for everything but cache reads, which cost half as much. Claude Opus 5.5’s token prices are lower than Claude Opus 5’s, most of all for cache reads. Like Claude Sonnet 5, the frontier tier gained through price: most of Claude Fable 5.1’s saving is the lower cache-read price.

934934 * **Cost without caching, priced from recorded token counts:** Anthropic re-priced the recorded token counts of the Claude Opus 5.5 and Claude Sonnet 5.5 runs, over the 478 problems, for the caching factors in [Why caching comes first](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#why-caching-comes-first).

935 * **The factors, by model:** Each factor is the token cost with every input token at the input price, output tokens unchanged, divided by the cost priced as a customer's organization is metered: for Claude Opus 5.5, 3.3 at `medium`, 2.4 at `low`, 3.8 at `high`, and 5.4 at `xhigh` (one run); for Claude Sonnet 5.5, with cache reads at $0.10 USD per million tokens, 3.1 at `high` and 2.4 at `medium`, or 2.8 at `high` at the launch price of $0.20 USD. Unlike the caching chart's runs, these incurred no web-search fees, and a task's first request is priced as a cache write.

935 * **The factors, by model:** Each factor is the token cost with every input token at the input price, output tokens unchanged, divided by the cost priced as a customer’s organization is metered: for Claude Opus 5.5, 3.3 at `medium`, 2.4 at `low`, 3.8 at `high`, and 5.4 at `xhigh` (one run); for Claude Sonnet 5.5, with cache reads at $0.10 USD per million tokens, 3.1 at `high` and 2.4 at `medium`, or 2.8 at `high` at the launch price of $0.20 USD. Unlike the caching chart’s runs, these incurred no web-search fees, and a task’s first request is priced as a cache write.

936936

9374. **BrowseComp:** Wei et al., "BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents," OpenAI, 2025. Effort figures use a 500-problem cut, one to three runs per setting, run August 3, 2026, with the default point pooling two runs from July 26 to 27, 2026. The measurement of routine work with a long cost tail in [Orchestrator strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#orchestrator-strategy-delegate-bulk-work) uses 10 problems that the model reliably solved on its own when the protocol was fixed, from a 26-problem slice, 50 delegated runs (5 per problem; August 1 to 2, 2026) and 70 runs of the model alone (50 from August 2 to 3, 2026; 20 archived from July 12 to 13 and August 1, 2026), $6.45 USD compared with $11.99 USD per run in expectation; delegated figures carry a measurement band of about 20%. The 10 problems are those on which the delegated configuration, a Claude Fable 5 orchestrator with one Claude Sonnet 5 worker, was right in all 5 runs and Claude Fable 5 alone in both archived runs; the delegated runs are those same runs, and only Claude Fable 5 alone was run again on the ten. Over its 70 runs, archived and new, Claude Fable 5 alone was right in 68. On those problems the delegated configuration cost about half as much as Claude Fable 5 alone on average (the $6.45 USD and $11.99 USD given earlier) and about a third as much at the 90th percentile ($12 USD compared with $33 USD per run), and its most expensive run cost $42 USD compared with $84 USD; that $84 USD run was also wrong. On the full, harder BrowseComp set, Claude Fable 5 alone reached the accuracy of the orchestrator configuration measured on that set at 22% to 30% lower cost.

9374. **BrowseComp:** Wei et al., “BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents,” OpenAI, 2025. Effort figures use a 500-problem cut, one to three runs per setting, run August 3, 2026, with the default point pooling two runs from July 26 to 27, 2026. The measurement of routine work with a long cost tail in [Orchestrator strategy](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#orchestrator-strategy-delegate-bulk-work) uses 10 problems that the model reliably solved on its own when the protocol was fixed, from a 26-problem slice, 50 delegated runs (5 per problem; August 1 to 2, 2026) and 70 runs of the model alone (50 from August 2 to 3, 2026; 20 archived from July 12 to 13 and August 1, 2026), $6.45 USD compared with $11.99 USD per run in expectation; delegated figures carry a measurement band of about 20%. The 10 problems are those on which the delegated configuration, a Claude Fable 5 orchestrator with one Claude Sonnet 5 worker, was right in all 5 runs and Claude Fable 5 alone in both archived runs; the delegated runs are those same runs, and only Claude Fable 5 alone was run again on the ten. Over its 70 runs, archived and new, Claude Fable 5 alone was right in 68. On those problems the delegated configuration cost about half as much as Claude Fable 5 alone on average (the $6.45 USD and $11.99 USD given earlier) and about a third as much at the 90th percentile ($12 USD compared with $33 USD per run), and its most expensive run cost $42 USD compared with $84 USD; that $84 USD run was also wrong. On the full, harder BrowseComp set, Claude Fable 5 alone reached the accuracy of the orchestrator configuration measured on that set at 22% to 30% lower cost.

938938

9395. **Agent-architecture scaling:** Kim et al., "Towards a Science of Scaling Agent Systems," arXiv:2512.08296, 2025. Independent external study, cited only for the direction of the finding on when delegation does not pay, not for any figure.

9395. **Agent-architecture scaling:** Kim et al., “Towards a Science of Scaling Agent Systems,” arXiv:2512.08296, 2025. Independent external study, cited only for the direction of the finding on when delegation does not pay, not for any figure.

940940

9416. **DeepWideSearch:** "DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking," arXiv:2510.20168, 2025. The 220 questions span 15 domains, each combining many-row collection with multi-hop retrieval; measured on the benchmark's standing row set, 3 runs per configuration, run August 2, 2026 (the single-worker team point ran July 26 to 27, 2026). With Claude Fable 5, `low` effort matched an orchestrator with a Claude Sonnet 5 worker at 29% lower cost, and took 4.5 minutes per problem, compared with 7.9 minutes at the default.

9416. **DeepWideSearch:** “DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking,” arXiv:2510.20168, 2025. The 220 questions span 15 domains, each combining many-row collection with multi-hop retrieval; measured on the benchmark’s standing row set, 3 runs per configuration, run August 2, 2026 (the single-worker team point ran July 26 to 27, 2026). With Claude Fable 5, `low` effort matched an orchestrator with a Claude Sonnet 5 worker at 29% lower cost, and took 4.5 minutes per problem, compared with 7.9 minutes at the default.

942942

9437. **DeepResearch Bench II:** Li et al., "DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Report," arXiv:2601.08536, 2026.

9437. **DeepResearch Bench II:** Li et al., “DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Report,” arXiv:2601.08536, 2026.

944944

945 * **What was measured:** Its 132 research tasks across 22 domains are graded against expert-derived binary rubrics. Measured on a 50-task subset stratified across all themes, one attempt per task, 3 runs per setting, on [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/overview) with the platform's own web search and fetch tools (August 26 to 27, 2026).

946 * **How scored:** Scored on the 33 tasks no configuration refused, with attempts the production safety classifiers cut short removed. Claude Opus 4.6 judges under the benchmark's rubric protocol; the original uses a different judge, and an Anthropic judge may favor the house style.

947 * **Scores:** Scores are each model's mean on the 33-task basis with its own pre-empted tasks removed; on the 21 of the 33 tasks where no configuration had an attempt cut short, Claude Fable 5.1 holds a 2-to-3-point lead over Claude Fable 5 at every effort level and both models are flat across effort.

948 * **How priced:** Costs are priced as a customer's organization is metered: each request's prior prompt as a cache read and its new tokens as a 5-minute cache write, a session's first request keeping the cache read it got, from the runs' own usage records, plus the platform's web-search fees. Each cost per task is a 50-task run's cost divided by 50, with the attempts that were cut short included, averaged over the three runs.

945 * **What was measured:** Its 132 research tasks across 22 domains are graded against expert-derived binary rubrics. Measured on a 50-task subset stratified across all themes, one attempt per task, 3 runs per setting, on [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/overview) with the platform’s own web search and fetch tools (August 26 to 27, 2026).

946 * **How scored:** Scored on the 33 tasks no configuration refused, with attempts the production safety classifiers cut short removed. Claude Opus 4.6 judges under the benchmark’s rubric protocol; the original uses a different judge, and an Anthropic judge may favor the house style.

947 * **Scores:** Scores are each model’s mean on the 33-task basis with its own pre-empted tasks removed; on the 21 of the 33 tasks where no configuration had an attempt cut short, Claude Fable 5.1 holds a 2-to-3-point lead over Claude Fable 5 at every effort level and both models are flat across effort.

948 * **How priced:** Costs are priced as a customer’s organization is metered: each request’s prior prompt as a cache read and its new tokens as a 5-minute cache write, a session’s first request keeping the cache read it got, from the runs’ own usage records, plus the platform’s web-search fees. Each cost per task is a 50-task run’s cost divided by 50, with the attempts that were cut short included, averaged over the three runs.

949949 * **Results, Claude Fable 5:** Claude Fable 5 cost $2.30 USD per task at `low` and $4.71 USD at `high`, priced the same way; the percentages in [Upgrade the model](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#upgrade-the-model) come from unrounded costs.

950950 * **Results, Claude Opus 5:** Claude Opus 5 at its default effort ran on the same surface and subset, three runs, on August 28, 2026: 68.8% on the raw 50 tasks, 70.8% on the 33-task basis, and 71.1% on the 21-task set, at $6.28 USD per task ($23.72 USD without caching); none of its attempts was cut short by the safety classifiers, under a safeguards deployment newer than the one the other models ran under.

951951 * **Compare models:** The research comparison in [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task) rests on these figures: Claude Fable 5.1 at `low` scored 10 points above Claude Sonnet 5 on the 33-task basis (66% against 56%) at about four times the cost per task over all 50 tasks ($4.48 USD against $1.18 USD); Claude Opus 5 at its default, whose scores and cost are given earlier in this reference, scored above Claude Fable 5.1 at its default, which scored 65% on the 33-task basis and cost $6.89 USD per task over all 50 tasks, so on research, Claude Fable 5.1 earns its price only at `low`.

952952 * **Caching chart:** The caching chart re-prices the same requests with every input token at the uncached rate.

953953

9548. **Corpus defect sweep:** Anthropic-internal, for work larger than one context window: a 21.6-million-token corpus from 14 public Python package sources with 130 planted defects and deterministic grading; protocol fixed before the runs and internally reviewed; three runs per configuration. Every configuration ran on Claude Managed Agents. The charted team configuration is one in which the Claude Fable 5.1 orchestrator ran the whole sweep inside the platform at its documented limit of 25 concurrent Claude Sonnet 5 workers, run August 30, 2026; its three runs scored F1 0.764, 0.825, and 0.791 after the extras audit (raw 0.751, 0.821, and 0.781) for $225 USD, $234 USD, and $283 USD. The team configuration cost about half as much as Claude Fable 5.1 alone at `low`, `medium`, or `high` (47% to 55% less), scored 10 to 12 points below it, and took about 2.3 hours per run, compared with 15 to 20 hours. The team configuration read about 560 million cached tokens per run, nearly all of them at Claude Sonnet 5's cache-read rate. That is about one and a half times the roughly 365 million that Claude Fable 5.1 read alone. Claude Fable 5.1 alone at `high` effort still holds peak accuracy, at about 2.2 times the team configuration's cost, so delegation here buys most of the accuracy, not all of it. The configuration with Claude Sonnet 5 alone ran August 3 to 4, 2026; the configurations with Claude Fable 5.1 alone ran August 24 to 25, 2026, under the platform's launch serving settings, three seeds per effort setting, on the same corpus build. The sandbox image carried installed copies of part of the corpus, and Claude Fable 5.1's final assembly step compared against them in 7 of 9 runs; re-grading without those additions moved the affected seeds by up to 3 points. Absolute F1 is specific to this corpus build, not comparable across benchmarks; configuration comparisons are like for like.

9548. **Corpus defect sweep:** Anthropic-internal, for work larger than one context window: a 21.6-million-token corpus from 14 public Python package sources with 130 planted defects and deterministic grading; protocol fixed before the runs and internally reviewed; three runs per configuration. Every configuration ran on Claude Managed Agents. The charted team configuration is one in which the Claude Fable 5.1 orchestrator ran the whole sweep inside the platform at its documented limit of 25 concurrent Claude Sonnet 5 workers, run August 30, 2026; its three runs scored F1 0.764, 0.825, and 0.791 after the extras audit (raw 0.751, 0.821, and 0.781) for $225 USD, $234 USD, and $283 USD. The team configuration cost about half as much as Claude Fable 5.1 alone at `low`, `medium`, or `high` (47% to 55% less), scored 10 to 12 points below it, and took about 2.3 hours per run, compared with 15 to 20 hours. The team configuration read about 560 million cached tokens per run, nearly all of them at Claude Sonnet 5’s cache-read rate. That is about one and a half times the roughly 365 million that Claude Fable 5.1 read alone. Claude Fable 5.1 alone at `high` effort still holds peak accuracy, at about 2.2 times the team configuration’s cost, so delegation here buys most of the accuracy, not all of it. The configuration with Claude Sonnet 5 alone ran August 3 to 4, 2026; the configurations with Claude Fable 5.1 alone ran August 24 to 25, 2026, under the platform’s launch serving settings, three seeds per effort setting, on the same corpus build. The sandbox image carried installed copies of part of the corpus, and Claude Fable 5.1’s final assembly step compared against them in 7 of 9 runs; re-grading without those additions moved the affected seeds by up to 3 points. Absolute F1 is specific to this corpus build, not comparable across benchmarks; configuration comparisons are like for like.

955955

9569. **GPQA Diamond:** Rein et al., "GPQA: A Graduate-Level Google-Proof Q\&A Benchmark," 2023. The 198-question Diamond subset, model-graded against reference answers, advisor tokens metered per request. Claude Haiku 5.5 and Claude Sonnet 5.5, each alone and with a Claude Opus 5.5 advisor, and Claude Opus 5.5 alone ran two runs per configuration on October 7, 2026, all with [server-side fallback](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback#server-side-fallback) off, at each model's default effort, and with refusals counted as wrong. A platform safety check refused one biology question per run on Claude Haiku 5.5 and Claude Sonnet 5.5 alone, one in two runs on Haiku 5.5 with the advisor, none on Sonnet 5.5 with the advisor, and six per run on Opus 5.5, which scored 91% with them counted as wrong and 93.5% on the questions it answered. Claude Haiku 5.5 alone scored 85%, 6 points below Opus 5.5. The advisor configurations added the tool without a prompt asking for consultations, and neither executor called the advisor on any question. The pairings scored 0.5 points (Claude Haiku 5.5) and 1.0 points (Claude Sonnet 5.5) above the executor alone, both within run-to-run noise.

9569. **GPQA Diamond:** Rein et al., “GPQA: A Graduate-Level Google-Proof Q\&A Benchmark,” 2023. The 198-question Diamond subset, model-graded against reference answers, advisor tokens metered per request. Claude Haiku 5.5 and Claude Sonnet 5.5, each alone and with a Claude Opus 5.5 advisor, and Claude Opus 5.5 alone ran two runs per configuration on October 7, 2026, all with [server-side fallback](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback#server-side-fallback) off, at each model’s default effort, and with refusals counted as wrong. A platform safety check refused one biology question per run on Claude Haiku 5.5 and Claude Sonnet 5.5 alone, one in two runs on Haiku 5.5 with the advisor, none on Sonnet 5.5 with the advisor, and six per run on Opus 5.5, which scored 91% with them counted as wrong and 93.5% on the questions it answered. Claude Haiku 5.5 alone scored 85%, 6 points below Opus 5.5. The advisor configurations added the tool without a prompt asking for consultations, and neither executor called the advisor on any question. The pairings scored 0.5 points (Claude Haiku 5.5) and 1.0 points (Claude Sonnet 5.5) above the executor alone, both within run-to-run noise.

957957

95810. **DeepSWE:** Datacurve, "DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks," arXiv:2607.07946, 2026. The set has 113 original tasks across five languages with program-based verifiers. Pairings are two runs each, run August 7, 2026, with advisor tokens metered per request, and used a client-side advisor loop rather than the advisor tool, with identical accounting. Single-model effort sweeps are single runs priced from token counts, a cache-aware approximation. Costs per task are run totals divided by 113. On this benchmark a low-effort Claude Sonnet 5 executor with an advisor scored 23 points above the same executor alone.

95810. **DeepSWE:** Datacurve, “DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks,” arXiv:2607.07946, 2026. The set has 113 original tasks across five languages with program-based verifiers. Pairings are two runs each, run August 7, 2026, with advisor tokens metered per request, and used a client-side advisor loop rather than the advisor tool, with identical accounting. Single-model effort sweeps are single runs priced from token counts, a cache-aware approximation. Costs per task are run totals divided by 113. On this benchmark a low-effort Claude Sonnet 5 executor with an advisor scored 23 points above the same executor alone.

959959

96096011. **Internal agentic-coding benchmark:**

961961

962 * **What was measured:** Anthropic-internal: 370 repository tasks graded by the repositories' own tests. The API figures were measured with a plain API agent and a 128,000-token output cap, one run per configuration: Opus 5 alone at the default effort August 9 to 10, 2026, and at `low` and `medium` August 10, 2026; Claude Fable 5.1 alone at five explicitly set effort values August 20, 2026 (the chart shows three of them); and the pairing August 24 to 25, 2026.

962 * **What was measured:** Anthropic-internal: 370 repository tasks graded by the repositories’ own tests. The API figures were measured with a plain API agent and a 128,000-token output cap, one run per configuration: Opus 5 alone at the default effort August 9 to 10, 2026, and at `low` and `medium` August 10, 2026; Claude Fable 5.1 alone at five explicitly set effort values August 20, 2026 (the chart shows three of them); and the pairing August 24 to 25, 2026.

963963 * **August runs:** The August runs had five attempts per task for the pairing and the Claude Opus 5 control and one for the other points.

964 * **How priced:** Costs are per attempt. Costs are priced as a customer's organization is metered: each agent-loop request's prior prompt as a cache read and its new tokens as a 5-minute cache write, from the runs' own usage records, and each advisor call, which uses no cache, from its recorded tokens, all at list prices.

964 * **How priced:** Costs are per attempt. Costs are priced as a customer’s organization is metered: each agent-loop request’s prior prompt as a cache read and its new tokens as a 5-minute cache write, from the runs’ own usage records, and each advisor call, which uses no cache, from its recorded tokens, all at list prices.

965965 * **Claude Opus 5.5 alone:** Claude Opus 5.5 alone ran on all 370 tasks, September 19 to 20, 2026: at its default effort (`medium`) and at `high` with five attempts per task, and at `low` and `xhigh` with one (369 of 370 scored at each, after a setup-check failure). Claude Opus 5.5 alone solved 74.8% of attempts at `low` for $0.35 USD per attempt, 86.6% at `medium` for $0.84 USD, 88.4% at `high` for $1.38 USD, and 91.1% at `xhigh` for $4.11 USD.

966966 * **Claude Opus 5.5 with a Claude Fable 5.1 advisor, runs:** The Claude Opus 5.5 executor at `high` with the released Claude Fable 5.1 as advisor (the August runs used a pre-release snapshot) ran five attempts per task on the same dates; one task failed its setup check, so 1,845 attempts were scored. The 279 attempts in which the advisor was turned away under load were re-run, and attempts whose consults timed out were kept, as in August.

967967 * **Claude Fable 5.1 alone:** For comparison, the released Claude Fable 5.1 alone at `high`, its default, scored 85.7% (317 of 370 tasks), one attempt per task on October 7, 2026.

968968 * **Consultations:** The August pairing averaged about two advisor consultations per attempt; the Claude Opus 5.5 pairing requested 1.39 and received 1.35.

969 * **Claude Opus 5.5 with a Claude Fable 5.1 advisor, results:** The Claude Opus 5.5 pairing scored 90.1% at $2.92 USD per attempt: compared with Claude Opus 5.5 at its default, `medium`, that is 3.5 points for about 3.5 times the money. The latency cost is the consultations themselves: about one or two extra frontier-model calls per task, each on the task's critical path. With the Claude Opus 5.5 executor, the advice saved almost no executor cost: $1.36 USD per attempt compared with $1.38 USD for Claude Opus 5.5 alone at `high`, while the consultations cost $1.55 USD.

969 * **Claude Opus 5.5 with a Claude Fable 5.1 advisor, results:** The Claude Opus 5.5 pairing scored 90.1% at $2.92 USD per attempt: compared with Claude Opus 5.5 at its default, `medium`, that is 3.5 points for about 3.5 times the money. The latency cost is the consultations themselves: about one or two extra frontier-model calls per task, each on the task’s critical path. With the Claude Opus 5.5 executor, the advice saved almost no executor cost: $1.36 USD per attempt compared with $1.38 USD for Claude Opus 5.5 alone at `high`, while the consultations cost $1.55 USD.

970970 * **August pairing:** In August, with a Claude Opus 5 executor and a Claude Fable 5.1 advisor, the executor spent $1.26 USD less per attempt than Claude Opus 5 alone at its default, and the consultations cost $2.47 USD, so that saving paid for about half of the advice. That August pairing was the most accurate configuration measured at the time, at $6.21 USD per attempt.

971 * **Claude Sonnet 5.5 alone, runs:** Claude Sonnet 5.5 alone ran on all 370 tasks on September 24, 2026, at its default effort (`high`) and at `medium`, one attempt per task, with the same 128,000-token output cap, which cut off none of its attempts. All 370 were scored at `medium`, and 369 of 370 at `high`, after a setup-check failure. Its costs are priced as a customer's organization is metered.

972972 * **Claude Sonnet 5.5 alone, results:** With no advisor, Claude Sonnet 5.5 scored 77.8% at its default, `high`, for $0.33 USD per attempt, about 9 points below Claude Opus 5.5 at its default for about two fifths of the cost; at `medium`, it scored 62.7% for $0.16 USD. At `medium`, Claude Sonnet 5.5 scored about 15 points lower than at its default, `high`, for about 48% of the cost per attempt.

973973 * **Compare models:** In [Compare models](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#compare-models-on-cost-per-task), Claude Opus 5.5 at its default cost under a third as much per attempt as Claude Fable 5.1 at `medium`, which is a single run.

974974

from line 980

980980 * **Claude Opus 5.5 runs:** The Claude Opus 5.5 16,384-token figure averages two runs (134 and 135 tasks scored), and its 64,000 and 128,000 figures are single runs (135 tasks each); two attempts in each 16,384-token run and one in each of the 64,000- and 128,000-token runs ended in a safety refusal and count as failures. Claude Opus 5.5 solved 57.6% at 16,384 (58.2% and 57.0%), 75.6% at 64,000, and 68.9% at 128,000. None of its attempts was cut off at 64,000 or 128,000, so the cap does not explain the difference between those two runs.

981981 * **Attempts the cap ended:** Only 1 of the 66 Claude Opus 5.5 attempts that the 16,384-token cap ended, and 9 of the 117 Claude Fable 5.1 attempts, still passed.

982982 * **Cost per attempt:** Cost per attempt was $3 USD at 16,384 and $4 USD at 64,000 for Claude Opus 5.5, at list prices, and $8 USD and $13 USD for Claude Fable 5.1, as a customer is billed.

983 * **SWE-bench Pro cap figures:** The SWE-bench Pro cap figures are one Claude Fable 5.1 run per cap at the default effort, run August 26, 2026, on a 100-problem subset stratified from reference 3's 482-problem set, not comparable to its scores; at the default effort the runs at 16,384 and at 64,000 each solved 94 of the 100 problems.

984 * **Turn length:** The per-turn distributions in the section's dot plot come from the Claude Opus 5.5 and Claude Fable 5.1 runs at 128,000: no Opus 5.5 turn reached the cap (the longest was about 61,000 tokens, and 0.56% of its turns exceeded 16,384), and one Fable 5.1 turn reached 128,000 (0.46% of its turns exceeded 16,384). With the cap at 128,000, 2 of Claude Fable 5.1's 14,331 turns were longer than 64,000 tokens, and no Claude Opus 5.5 turn was.

983 * **SWE-bench Pro cap figures:** The SWE-bench Pro cap figures are one Claude Fable 5.1 run per cap at the default effort, run August 26, 2026, on a 100-problem subset stratified from reference 3’s 482-problem set, not comparable to its scores; at the default effort the runs at 16,384 and at 64,000 each solved 94 of the 100 problems.

984 * **Turn length:** The per-turn distributions in the section’s dot plot come from the Claude Opus 5.5 and Claude Fable 5.1 runs at 128,000: no Opus 5.5 turn reached the cap (the longest was about 61,000 tokens, and 0.56% of its turns exceeded 16,384), and one Fable 5.1 turn reached 128,000 (0.46% of its turns exceeded 16,384). With the cap at 128,000, 2 of Claude Fable 5.1’s 14,331 turns were longer than 64,000 tokens, and no Claude Opus 5.5 turn was.

985985

98613. **Chartography:** Surge AI, "Chartography," 2026.

98613. **Chartography:** Surge AI, “Chartography,” 2026.

987987

988 * **What was measured:** The complete released 100-question set, measured August 6 and 9, 2026 (Claude Opus 5 alone) and September 20, 2026 (Claude Opus 5.5), with Anthropic's implementation on Claude Managed Agents (standard cloud sandbox; advisor configurations use the Managed Agents advisor).

988 * **What was measured:** The complete released 100-question set, measured August 6 and 9, 2026 (Claude Opus 5 alone) and September 20, 2026 (Claude Opus 5.5), with Anthropic’s implementation on Claude Managed Agents (standard cloud sandbox; advisor configurations use the Managed Agents advisor).

989989 * **How scored:** Claude Sonnet 4.6 grades instead of the reference judge and the benchmark runs with tools, so scores compare across configurations here but not to the published leaderboard. They are also not comparable to the Chartography results in the Claude Opus 5.5 system card, which use a different grader and run at `max` effort. Two runs per configuration (three for Claude Opus 5.5), pooled; run-to-run spreads were up to 10 points.

990 * **How priced:** Costs are what a customer running the agent routinely is billed: each chart's first request reads the agent's shared system prompt and tools from the cache, as it does when another session of the same agent ran in the previous 5 minutes. A chart run on its own costs about $0.03 USD more with Claude Opus 5 or Claude Opus 5.5 and about $0.12 USD more with Claude Fable 5.1. The August figures are re-priced this way from the runs' usage records. Costs exclude sandbox time, which added under 1% to the August runs.

991 * **Claude Fable 5.1 alone, August runs:** The runs of Claude Fable 5.1 alone are from August 24, 2026, under the platform's launch serving settings, two runs per setting; six attempts hit the 15-minute session cap and score 0, and two charts per run were answered by Claude Opus 5 after a safety refusal.

992 * **Claude Opus 5 with a Claude Fable 5.1 advisor:** The Claude Opus 5 low-effort executor with a Claude Fable 5.1 advisor ran twice on August 30, 2026, under the same settings (63.0 and 67.0, mean 65.0, at $0.47 USD a chart; the advisor was consulted on 88% of tasks in each run, and 4 of its 219 replies came from Claude Opus 5 instead, each after a production safety filter stopped the advisor's own reply). That pairing matched Claude Fable 5.1 alone at `medium` within run-to-run noise (65.0 compared with 67.5) at about 1.8 times the cost per task: a consult that fires on most tasks costs more than running the stronger model itself.

990 * **How priced:** Costs are what a customer running the agent routinely is billed: each chart’s first request reads the agent’s shared system prompt and tools from the cache, as it does when another session of the same agent ran in the previous 5 minutes. A chart run on its own costs about $0.03 USD more with Claude Opus 5 or Claude Opus 5.5 and about $0.12 USD more with Claude Fable 5.1. The August figures are re-priced this way from the runs’ usage records. Costs exclude sandbox time, which added under 1% to the August runs.

991 * **Claude Fable 5.1 alone, August runs:** The runs of Claude Fable 5.1 alone are from August 24, 2026, under the platform’s launch serving settings, two runs per setting; six attempts hit the 15-minute session cap and score 0, and two charts per run were answered by Claude Opus 5 after a safety refusal.

992 * **Claude Opus 5 with a Claude Fable 5.1 advisor:** The Claude Opus 5 low-effort executor with a Claude Fable 5.1 advisor ran twice on August 30, 2026, under the same settings (63.0 and 67.0, mean 65.0, at $0.47 USD a chart; the advisor was consulted on 88% of tasks in each run, and 4 of its 219 replies came from Claude Opus 5 instead, each after a production safety filter stopped the advisor’s own reply). That pairing matched Claude Fable 5.1 alone at `medium` within run-to-run noise (65.0 compared with 67.5) at about 1.8 times the cost per task: a consult that fires on most tasks costs more than running the stronger model itself.

993993 * **Claude Opus 5.5 runs:** Claude Opus 5.5 ran at `low`, with server-side fallback off and a safety classifier judging every tool call: three runs alone (70, 68, and 68) and three with a Claude Fable 5.1 advisor configured (59, 63, and 63), in which it consulted the advisor on 1 of 300 tasks.

994994 * **Results at `low` effort:** On this chart-reading benchmark, Claude Opus 5.5 at `low` scored 68.7 for about $0.03 USD a chart, against 62.5 for $0.15 USD from Claude Fable 5.1 at `low` and 49 for $0.16 USD from Claude Opus 5 at `low`.

995995 * **Claude Fable 5.1 alone, October runs:** For comparison with the Claude Opus 5.5 pairing, Claude Fable 5.1 alone at `high`, its default, was run three times on October 7, 2026, with the same implementation and platform settings as the Claude Opus 5.5 runs: 82, 79, and 79, a mean of 80.0.

from line 999

999999

10001000 * **What was measured:** An Anthropic-constructed set of 44 support tickets with deterministic grading, run in early August 2026 and reported on August 8, 2026, under six system prompts, each adding to the same clean prompt one pattern common in prompts written for Claude Opus 4.8 and Claude Sonnet 4.6.

10011001 * **The chart:** Each chart point is one of three cases (older model, newer model on the same prompt, newer model after the audit) averaged over the six prompts and 44 tickets.

1002 * **Results, Claude Opus 5:** On Claude Opus 5, the prompts written for Claude Opus 4.8 cost 36% more per ticket than they did on Claude Opus 4.8, for no change in accuracy. After the audit they cost 14% less on Claude Opus 5 than before it (from the unrounded means; the chart's rounded costs, 3.43 and 2.93 US cents per ticket, give 14.6%), and Claude Opus 5 resolved 97% of tickets correctly, up from 92%. By pattern, on Claude Opus 5: removing "verify twice" cut the cost per ticket by a third, and removing "be maximally thorough" cut it almost as much; removing the retired thinking setting, the contradictory rules, and the hand-rolled scratchpad each restored 7 to 11 points of accuracy.

1002 * **Results, Claude Opus 5:** On Claude Opus 5, the prompts written for Claude Opus 4.8 cost 36% more per ticket than they did on Claude Opus 4.8, for no change in accuracy. After the audit they cost 14% less on Claude Opus 5 than before it (from the unrounded means; the chart’s rounded costs, 3.43 and 2.93 US cents per ticket, give 14.6%), and Claude Opus 5 resolved 97% of tickets correctly, up from 92%. By pattern, on Claude Opus 5: removing “verify twice” cut the cost per ticket by a third, and removing “be maximally thorough” cut it almost as much; removing the retired thinking setting, the contradictory rules, and the hand-rolled scratchpad each restored 7 to 11 points of accuracy.

10031003 * **Results, Claude Sonnet 5:** On the Claude Sonnet 4.6 to Claude Sonnet 5 migration, the audit took 14% off the cost per ticket at the same accuracy.

10041004 * **Caveats:** The Opus 5 accuracy gain has a 95% confidence interval of 3 to 8 points; the Sonnet accuracy differences are within noise.

10051005

100615. **Data-file question set:** An Anthropic-constructed set of 25 aggregate questions over a 1,862-row slice of a public liquor-sales CSV, with ground truth computed by pandas and exact-match grading, run on Claude Sonnet 5 and Claude Opus 5 with thinking disabled (the in-context arm cannot complete at the default), a 4,000-token output cap, and no prompt caching, three runs per configuration, run August 19, 2026. The file arm uploads the CSV through the Files API and uses the `code_execution_20260120` tool. The in-context arm's three Claude Sonnet 5 runs answered 6, 7, and 7 of the 25 questions correctly; the chart's label and the text give their mean, 6.67, to the nearest question, and the chart's point is drawn at the mean, 26.7%.

100615. **Data-file question set:** An Anthropic-constructed set of 25 aggregate questions over a 1,862-row slice of a public liquor-sales CSV, with ground truth computed by pandas and exact-match grading, run on Claude Sonnet 5 and Claude Opus 5 with thinking disabled (the in-context arm cannot complete at the default), a 4,000-token output cap, and no prompt caching, three runs per configuration, run August 19, 2026. The file arm uploads the CSV through the Files API and uses the `code_execution_20260120` tool. The in-context arm’s three Claude Sonnet 5 runs answered 6, 7, and 7 of the 25 questions correctly; the chart’s label and the text give their mean, 6.67, to the nearest question, and the chart’s point is drawn at the mean, 26.7%.

10071007

1008100816. **Cache duration measurement:**

10091009

10101010 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run on Claude Sonnet 5 on August 23, 2026, and on Claude Opus 5.5 on September 19 and 20, 2026, at its default effort (`medium`) and at `high`, on the Messages API with the same harness (for Claude Opus 5.5, a port of it that sends the same request bodies), the Claude Opus 5.5 cells with `max_tokens` raised to 4,096, with pauses inserted before a randomly chosen share of turns (none, 5%, 10%, and every turn at 6 minutes on all 20 issues on both models, plus every turn at 2 minutes on Claude Sonnet 5; 20-minute and 45-minute pauses on a 5-issue subset on both models). The Claude Opus 5.5 figures on this page cover both effort levels.

1011 * **Other models and dates:** Claude Opus 5's keep-alive figures below come from the same job on August 23, 2026, with `max_tokens` raised to 4,096, on the same schedules except the 2-minute and 45-minute pauses. Claude Sonnet 5.5 ran the same measurement as Claude Sonnet 5 (the same pause schedules and three cache settings, three runs per cell), with the ported harness used for Claude Opus 5.5, at its default effort, `high`, on September 24, 2026.

1012 * **How scored and priced:** Three runs per cell, cost computed from each response's `usage` fields at list prices (for Claude Opus 5.5, $4 USD input, $5 USD 5-minute write, $8 USD 1-hour write, $0.20 USD cache read, and $20 USD output per million tokens; Claude Sonnet 5 ran on an Anthropic-internal organization whose usage is metered the same way as a customer organization's), accuracy against the same gold labels.

1013 * **How run and priced, Claude Sonnet 5.5:** Claude Sonnet 5.5 set `max_tokens` to 4,096 on session requests and to 1 on keep-alive requests, ran on an Anthropic-internal organization whose usage is metered the same way as a customer organization's, and is priced at its current list prices ($2 USD input, $2.50 USD 5-minute write, $4 USD 1-hour write, $0.10 USD cache read, and $10 USD output per million tokens). Apart from the higher `max_tokens`, which no turn reached, one input differs from the Claude Sonnet 5 runs: the job's one animated GIF was sent as a PNG of its first frame, the only frame the API uses ([Vision](https://platform.claude.com/docs/en/build-with-claude/vision)), and the two count as the same number of input tokens.

1014 * **Break-even:** The break-even is about 3.3% of turns on Claude Sonnet 5 and 3.1% to 3.2% on Claude Opus 5.5: the median of each session's break-even share, computed by the cost model from that session's turn-by-turn context sizes, over all 45 Claude Sonnet 5 twenty-issue sessions and the 36 Claude Opus 5.5 twenty-issue sessions at each effort level (every pause schedule run on the full job, under all three cache settings, three runs each; the 5-issue cells are not in it). The break-even on Claude Sonnet 5.5 is about 3.1% of turns, the median over its 45 twenty-issue sessions. The cache duration chart plots these Claude Sonnet 5.5 sessions, three per point.

1015 * **5-minute and 1-hour settings, Claude Sonnet 5 and Claude Opus 5.5:** In the 5% cell the 5-minute and 1-hour settings tied on Claude Sonnet 5, because that draw's pauses fell on small prefixes; on Claude Opus 5.5 they nearly tied. On Claude Sonnet 5, at its cache-read price of $0.20 USD per million tokens, a session cost $0.60 USD, $0.69 USD, and $0.99 USD on the 5-minute setting with no pauses, 5%, and 10% of turns paused, and $0.70 USD, $0.70 USD, and $0.71 USD on the 1-hour setting.

1011 * **Other models and dates:** Claude Opus 5’s keep-alive figures below come from the same job on August 23, 2026, with `max_tokens` raised to 4,096, on the same schedules except the 2-minute and 45-minute pauses. Claude Sonnet 5.5 ran the same measurement as Claude Sonnet 5 (the same pause schedules and three cache settings, three runs per cell), with the ported harness used for Claude Opus 5.5, at its default effort, `high`, on September 24, 2026.

1012 * **How scored and priced:** Three runs per cell, cost computed from each response’s `usage` fields at list prices (for Claude Opus 5.5, $4 USD input, $5 USD 5-minute write, $8 USD 1-hour write, $0.20 USD cache read, and $20 USD output per million tokens; Claude Sonnet 5 ran on an Anthropic-internal organization whose usage is metered the same way as a customer organization’s), accuracy against the same gold labels.

1013 * **How run and priced, Claude Sonnet 5.5:** Claude Sonnet 5.5 set `max_tokens` to 4,096 on session requests and to 1 on keep-alive requests, ran on an Anthropic-internal organization whose usage is metered the same way as a customer organization’s, and is priced at its current list prices ($2 USD input, $2.50 USD 5-minute write, $4 USD 1-hour write, $0.10 USD cache read, and $10 USD output per million tokens). Apart from the higher `max_tokens`, which no turn reached, one input differs from the Claude Sonnet 5 runs: the job’s one animated GIF was sent as a PNG of its first frame, the only frame the API uses ([Vision](https://platform.claude.com/docs/en/build-with-claude/vision)), and the two count as the same number of input tokens.

1014 * **Break-even:** The break-even is about 3.3% of turns on Claude Sonnet 5 and 3.1% to 3.2% on Claude Opus 5.5: the median of each session’s break-even share, computed by the cost model from that session’s turn-by-turn context sizes, over all 45 Claude Sonnet 5 twenty-issue sessions and the 36 Claude Opus 5.5 twenty-issue sessions at each effort level (every pause schedule run on the full job, under all three cache settings, three runs each; the 5-issue cells are not in it). The break-even on Claude Sonnet 5.5 is about 3.1% of turns, the median over its 45 twenty-issue sessions. The cache duration chart plots these Claude Sonnet 5.5 sessions, three per point.

1015 * **5-minute and 1-hour settings, Claude Sonnet 5 and Claude Opus 5.5:** In the 5% cell the 5-minute and 1-hour settings tied on Claude Sonnet 5, because that draw’s pauses fell on small prefixes; on Claude Opus 5.5 they nearly tied. On Claude Sonnet 5, at its cache-read price of $0.20 USD per million tokens, a session cost $0.60 USD, $0.69 USD, and $0.99 USD on the 5-minute setting with no pauses, 5%, and 10% of turns paused, and $0.70 USD, $0.70 USD, and $0.71 USD on the 1-hour setting.

10161016 * **Keep-alive requests, Claude Sonnet 5 and Claude Opus 5:** Anthropic measured keep-alive requests that refresh the 5-minute cache on Claude Sonnet 5 and Claude Opus 5 on August 23, 2026, and on Claude Opus 5.5 in the runs above, always sent with `max_tokens: 1`. On Claude Sonnet 5 they cost 7.7% less than the 1-hour setting with 5% of turns paused and about the same with 10%; on Claude Opus 5 no difference was measurable at either share; on both they cost more with a pause of 6 minutes or more before every turn. So on those two models the setting to use is the 1-hour duration, not keep-alive requests.

1017 * **Keep-alive requests, Claude Opus 5.5:** On Claude Opus 5.5 keep-alive requests cost 8% to 18% less than the 1-hour setting with 5% and 10% of turns paused (8% to 13% less at its default effort, `medium`, and 10% to 18% less at `high`; about 10% to 15% once between-session noise is removed by re-billing each keep-alive session's own tokens at 1-hour cache prices), and more with a pause before every turn: 4% to 6% more at 6 minutes, 9% to 10% at 20 minutes, and 56% to 58% at 45 minutes.

1017 * **Keep-alive requests, Claude Opus 5.5:** On Claude Opus 5.5 keep-alive requests cost 8% to 18% less than the 1-hour setting with 5% and 10% of turns paused (8% to 13% less at its default effort, `medium`, and 10% to 18% less at `high`; about 10% to 15% once between-session noise is removed by re-billing each keep-alive session’s own tokens at 1-hour cache prices), and more with a pause before every turn: 4% to 6% more at 6 minutes, 9% to 10% at 20 minutes, and 56% to 58% at 45 minutes.

10181018 * **Requests with `max_tokens: 0`:** Measured October 20, 2026, on Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, three times per model, on a 10,000-token system prompt with the 5-minute cache. A `max_tokens: 0` request sent 4 minutes after the cache write returned no output, read the whole entry, and wrote nothing. A request 4 minutes later, 8 minutes after the write, still read the whole entry. Without the `max_tokens: 0` request, the request at 8 minutes read nothing and wrote the prefix again, all 12 times.

1019 * **5-minute and 1-hour settings, Claude Sonnet 5.5:** With 5% of turns paused the 5-minute setting cost 5% more than the 1-hour setting on average (2% less to 12% more across the three runs), which is within run-to-run variation: that draw's pauses fell on small prefixes, as on Claude Sonnet 5, and re-billing each 5-minute session's own tokens at 1-hour cache prices gives about 1% more. With 10% paused it cost 54% more (36% to 73% more). With no pauses, the 5-minute setting cost 19% less than the 1-hour setting on average (3% to 32% less across the three runs; the 1-hour setting's higher write price predicts about 20%), and 21% to 24% less in every run with 2-minute pauses.

1020 * **Keep-alive requests, Claude Sonnet 5.5:** Keep-alive requests cost 16% less than the 1-hour setting on average with 5% of turns paused (13% to 20% less across the three runs) and 8% less with 10% (3% to 10% less), and cost more with a pause of 6 minutes or more before every turn. With 5% and 10% of turns paused, the pauses lasted 6 to 32 minutes on Claude Sonnet 5.5 and on Claude Opus 5.5. Re-billing each keep-alive session's own tokens at 1-hour cache prices, as for Claude Opus 5.5 above, gives 15% to 16% less at 5% paused and about 10% less at 10%, and with a pause before every turn 7% more at 6 minutes, 9% to 10% more at 20 minutes, and 66% to 73% more at 45 minutes. The 6-minute figures leave out run 3, in which 2 of that session's 39 keep-alive requests timed out: the cache expired before those two turns, the session wrote 214,000 tokens to the cache against 80,000 to 85,000 in runs 1 and 2, and it cost 38% more than run 3's 1-hour session.

1021 * **Keep-alive requests, Claude Sonnet 5.5, by run:** Comparing each run's keep-alive session with the same run's 1-hour session instead, keep-alive cost from about the same to 10% more at 6 minutes (runs 1 and 2; 4% more on their average), from about the same to 16% more at 20 minutes, and 64% to 80% more at 45 minutes.

1022 * **Keep-alive requests, Claude Sonnet 5.5, sample size:** The keep-alive chart's 6-minute points average runs 1 and 2 for all three settings. Each cell is three sessions (two for the 6-minute figures above). With no pauses, keep-alive sends no extra requests and so matches the 5-minute setting, yet the two differed by as much as 23% within a single run, so a difference of a few percent in a three-run average is not a clear effect.

1019 * **5-minute and 1-hour settings, Claude Sonnet 5.5:** With 5% of turns paused the 5-minute setting cost 5% more than the 1-hour setting on average (2% less to 12% more across the three runs), which is within run-to-run variation: that draw’s pauses fell on small prefixes, as on Claude Sonnet 5, and re-billing each 5-minute session’s own tokens at 1-hour cache prices gives about 1% more. With 10% paused it cost 54% more (36% to 73% more). With no pauses, the 5-minute setting cost 19% less than the 1-hour setting on average (3% to 32% less across the three runs; the 1-hour setting’s higher write price predicts about 20%), and 21% to 24% less in every run with 2-minute pauses.

1020 * **Keep-alive requests, Claude Sonnet 5.5:** Keep-alive requests cost 16% less than the 1-hour setting on average with 5% of turns paused (13% to 20% less across the three runs) and 8% less with 10% (3% to 10% less), and cost more with a pause of 6 minutes or more before every turn. With 5% and 10% of turns paused, the pauses lasted 6 to 32 minutes on Claude Sonnet 5.5 and on Claude Opus 5.5. Re-billing each keep-alive session’s own tokens at 1-hour cache prices, as for Claude Opus 5.5 above, gives 15% to 16% less at 5% paused and about 10% less at 10%, and with a pause before every turn 7% more at 6 minutes, 9% to 10% more at 20 minutes, and 66% to 73% more at 45 minutes. The 6-minute figures leave out run 3, in which 2 of that session’s 39 keep-alive requests timed out: the cache expired before those two turns, the session wrote 214,000 tokens to the cache against 80,000 to 85,000 in runs 1 and 2, and it cost 38% more than run 3’s 1-hour session.

1021 * **Keep-alive requests, Claude Sonnet 5.5, by run:** Comparing each run’s keep-alive session with the same run’s 1-hour session instead, keep-alive cost from about the same to 10% more at 6 minutes (runs 1 and 2; 4% more on their average), from about the same to 16% more at 20 minutes, and 64% to 80% more at 45 minutes.

1022 * **Keep-alive requests, Claude Sonnet 5.5, sample size:** The keep-alive chart’s 6-minute points average runs 1 and 2 for all three settings. Each cell is three sessions (two for the 6-minute figures above). With no pauses, keep-alive sends no extra requests and so matches the 5-minute setting, yet the two differed by as much as 23% within a single run, so a difference of a few percent in a three-run average is not a clear effect.

10231023 * **Caching off and on, Claude Sonnet 5:** On Claude Sonnet 5, the same job was run with caching off and with caching on, three runs each on August 19, 2026, with `max_tokens` at 700: $3.36 USD against $0.57 USD a session on average, a saving of 83%.

1024 * **Priced from recorded token counts, Claude Sonnet 5.5:** Anthropic also re-priced the recorded token counts of the three Claude Sonnet 5.5 sessions with the 5-minute cache and no pauses for the caching figure in [Why caching comes first](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#why-caching-comes-first). Each of these three sessions is one conversation of 40 requests over the job's 20 issues. At list prices as of October 12, 2026, the sessions cost $0.51 USD each on average, from the usage fields, against $3.70 USD with every input token at the input price and output tokens unchanged: a saving of 86% (86% to 87% across the three sessions), or a factor of 7.3. At the launch cache-read price of $0.20 USD the saving is 82%, so the rise over the Claude Sonnet 5 figure comes from the lower cache-read price.

1024 * **Priced from recorded token counts, Claude Sonnet 5.5:** Anthropic also re-priced the recorded token counts of the three Claude Sonnet 5.5 sessions with the 5-minute cache and no pauses for the caching figure in [Why caching comes first](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#why-caching-comes-first). Each of these three sessions is one conversation of 40 requests over the job’s 20 issues. At list prices as of October 12, 2026, the sessions cost $0.51 USD each on average, from the usage fields, against $3.70 USD with every input token at the input price and output tokens unchanged: a saving of 86% (86% to 87% across the three sessions), or a factor of 7.3. At the launch cache-read price of $0.20 USD the saving is 82%, so the rise over the Claude Sonnet 5 figure comes from the lower cache-read price.

10251025 * **Priced from recorded token counts, Claude Opus 5.5:** Re-priced the same way, the three Claude Opus 5.5 sessions at its default effort, `medium`, with the 5-minute cache and no pauses cost $1.02 USD each on average against $6.93 USD, a saving of 85%, or a factor of 6.8; the model used thinking in 79% of requests.

1026 * **How thinking was set, two sets of runs only:** The requests of the three Claude Sonnet 5.5 sessions with the 5-minute cache and no pauses set no thinking option, so thinking was left at the model's default, and the model used it in 38% of requests. The requests of the Claude Sonnet 5 runs with caching off and with caching on also set no thinking option, so thinking was left at the model's default, and the model used it in 41 of 121 requests with caching off and 40 of 120 with caching on.

1026 * **How thinking was set, two sets of runs only:** The requests of the three Claude Sonnet 5.5 sessions with the 5-minute cache and no pauses set no thinking option, so thinking was left at the model’s default, and the model used it in 38% of requests. The requests of the Claude Sonnet 5 runs with caching off and with caching on also set no thinking option, so thinking was left at the model’s default, and the model used it in 41 of 121 requests with caching off and 40 of 120 with caching on.

10271027 * **Latency after a pause:** On Claude Sonnet 5 and Claude Opus 5, the turn after a pause kept its warm-cache latency on the 1-hour setting.

10281028 * **Changing the duration mid-conversation:** Measured October 21, 2026, on Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, on the Messages API with no tools, adaptive thinking, `high` effort, a cache breakpoint on the system prompt, and [automatic caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching) or an explicit breakpoint on the last block. The conversations held about 5,000 or 100,000 tokens of filler text, with fixed one-word assistant turns and no thinking blocks. In a conversation cached at 5 minutes, the first 1-hour request that added a turn read nothing and wrote the whole prompt at the 1-hour price (55 of 55). The next 1-hour request read everything (36 of 36). A system prompt cached at 1 hour from the first request was still read (12 of 12). A 1-hour request that added nothing read the 5-minute entries (31 of 31) and did not extend them: 8 minutes later, the same request read nothing (12 of 12). In a conversation cached at 1 hour, a 5-minute request read everything and wrote only its new turn (12 of 12). The requests ran in one Anthropic-internal organization on the Claude API.

10291029 * **Changing the duration mid-conversation, how priced:** From the `usage` fields at list prices, at just over 100,000 tokens, with `max_tokens` at 16, the first 1-hour request that added a turn cost $2.01 USD, $0.80 USD, $0.40 USD, and $0.10 USD on the four models in the order named. A turn that read the cache cost $0.026 USD, $0.020 USD, $0.010 USD, and $0.005 USD. Claude Haiku 5.5 is at its [long context prices](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing).

10301030

103117. **Cache-read share in production:** Aggregated first-party Claude API usage for the 14 days ending August 23, 2026, direct API product only, Anthropic-internal organizations excluded, no organization identified. An organization-day counts as an agent loop when its requests carry tool definitions and tool results, its prompts hold 9 or more prior tool calls on average, caching was used, and it made at least 10 such requests (the API has no conversation identifier, so this stands in for conversation length): 303,003 organization-days across 106,487 organizations, median cache-read share 84.2% of all input tokens, upper quartile 91.7%. Use-case labels (the organization's declared use case, or otherwise its classified one) cover 74% of those organization-days and 99% of their tokens; about 72% of coding organization-days read 80% or more. The top decile of organization-days reads 95.9% or more for coding and 94.2% to 94.8% for support, research, data, and other agents. The request-level split at 25 or more prior tool calls comes from a six-hour sample: coding 92% read, 7% write, under 1% uncached.

103117. **Cache-read share in production:** Aggregated first-party Claude API usage for the 14 days ending August 23, 2026, direct API product only, Anthropic-internal organizations excluded, no organization identified. An organization-day counts as an agent loop when its requests carry tool definitions and tool results, its prompts hold 9 or more prior tool calls on average, caching was used, and it made at least 10 such requests (the API has no conversation identifier, so this stands in for conversation length): 303,003 organization-days across 106,487 organizations, median cache-read share 84.2% of all input tokens, upper quartile 91.7%. Use-case labels (the organization’s declared use case, or otherwise its classified one) cover 74% of those organization-days and 99% of their tokens; about 72% of coding organization-days read 80% or more. The top decile of organization-days reads 95.9% or more for coding and 94.2% to 94.8% for support, research, data, and other agents. The request-level split at 25 or more prior tool calls comes from a six-hour sample: coding 92% read, 7% write, under 1% uncached.

10321032

1033103318. **Cache-break, compaction timing, and status-line measurements:**

10341034

1035 * **October 2026 test, one change to a long conversation:** Measured October 20, 2026, on Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, on the Messages API with no tools, adaptive thinking, each model's default effort, the 5-minute cache, a cache breakpoint on the system prompt, and [automatic caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching). The conversation held about 101,000 tokens: a 20,000-token system prompt, 13 user turns of filler text, the model's own reply to the first turn with its thinking block, and one-word replies to the rest. Once one request had cached the conversation, a second added one short turn, and each measured request sent that longer conversation with one change. Two conversations per model gave the same result.

1035 * **October 2026 test, one change to a long conversation:** Measured October 20, 2026, on Claude Fable 5.1, Claude Opus 5.5, Claude Sonnet 5.5, and Claude Haiku 5.5, on the Messages API with no tools, adaptive thinking, each model’s default effort, the 5-minute cache, a cache breakpoint on the system prompt, and [automatic caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching). The conversation held about 101,000 tokens: a 20,000-token system prompt, 13 user turns of filler text, the model’s own reply to the first turn with its thinking block, and one-word replies to the rest. Once one request had cached the conversation, a second added one short turn, and each measured request sent that longer conversation with one change. Two conversations per model gave the same result.

10361036 * **October 2026 test, tokens read and written:** With no change, and with a per-message effort change, the request read all 101,000 tokens and wrote none. With a 25-token mid-conversation system message, or a 25-token status line at the end of the newest user turn, it read them all and wrote only the new text. A top-level effort change (to `low` on Claude Fable 5.1 and Claude Sonnet 5.5, to `high` on Claude Opus 5.5 and Claude Haiku 5.5) read the 20,000-token system prompt and wrote 81,000 tokens. The same 25 tokens at the front or the end of the system prompt read nothing and wrote all 101,000. The request after a mid-conversation system message, a top-level effort change, or 25 tokens added to the end of the system prompt read the new prefix; a new status line at the front wrote everything again. A further test on October 21, 2026, on the same four models with conversations of 5,000 tokens, and on all but Claude Fable 5.1 with conversations of 100,000 tokens, all with one-word replies, sent a beta header for the first time mid-conversation, as a first per-message effort change may: that request read everything the request before it had cached, all 54 times. A system message sent with no beta header did the same, all 15 times.

1037 * **October 2026 test, how priced:** The chart gives the prompt side of one request, from its `usage` fields at list prices. The chart's values, in the order system prompt edit, system message, top-level effort change, and per-message effort change: Claude Fable 5.1 $1.26 USD, $0.026 USD, $1.02 USD, and $0.025 USD; Claude Opus 5.5 $0.50 USD, $0.020 USD, $0.41 USD, and $0.020 USD; Claude Sonnet 5.5 $0.25 USD, $0.010 USD, $0.20 USD, and $0.010 USD. The unchanged request repeated an earlier request byte for byte; a new turn of about 25 tokens adds under $0.001 USD. On Claude Haiku 5.5, whose requests were over 100,000 prompt tokens and so at its [long context prices](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing): $0.063 USD for the system prompt edit, $0.052 USD for the effort change, and $0.005 USD for the read. The requests ran on an Anthropic-internal organization whose usage is metered the same way as a customer organization's, and on which the API does not enforce the [prefix check](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#prefix-check) by default, so its system prompt edits were accepted. On accounts created on or after August 31, 2026, 00:00 UTC, a request to one of these four models that sends earlier thinking blocks back gets a 400 error by default if it changes `system`, `tools`, or a message before one of them.

1038 * **What was measured:** The triage agent's long variant from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run August 24, 2026, on Claude Sonnet 5 with the 5-minute cache, cost from the usage fields at list prices, five sessions per arm: a no-change arm at the default effort throughout ($0.81 USD per session), and two arms that start at low effort and make the same two cache-breaking changes, a switch to the default effort and one added tool, either mid-session at requests 12 and 17 ($0.95 USD) or together on the first request after the first compaction ($0.75 USD).

1039 * **Fourth arm:** A fourth arm of six sessions, run August 25, 2026, made the same two changes on the request that triggered the first compaction ($0.92 USD per session): that request's summarization pass wrote the 81,000-token context to the cache instead of reading it, so that pass cost $0.21 USD against $0.04 USD for the same pass in the boundary arm.

1040 * **Compaction:** Sessions first compacted at request 21 to 25 (16 of the 21 sessions at request 22), once the prompt passed the 80,000-token compaction trigger, and two no-change sessions compacted a second time near the end. The boundary arm's lower total than the no-change arm reflects its low-effort requests before the change and those second compactions rather than caching: the two arms' re-write costs differ by under $0.01 USD.

1037 * **October 2026 test, how priced:** The chart gives the prompt side of one request, from its `usage` fields at list prices. The chart’s values, in the order system prompt edit, system message, top-level effort change, and per-message effort change: Claude Fable 5.1 $1.26 USD, $0.026 USD, $1.02 USD, and $0.025 USD; Claude Opus 5.5 $0.50 USD, $0.020 USD, $0.41 USD, and $0.020 USD; Claude Sonnet 5.5 $0.25 USD, $0.010 USD, $0.20 USD, and $0.010 USD. The unchanged request repeated an earlier request byte for byte; a new turn of about 25 tokens adds under $0.001 USD. On Claude Haiku 5.5, whose requests were over 100,000 prompt tokens and so at its [long context prices](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing): $0.063 USD for the system prompt edit, $0.052 USD for the effort change, and $0.005 USD for the read. The requests ran on an Anthropic-internal organization whose usage is metered the same way as a customer organization’s, and on which the API does not enforce the [prefix check](https://platform.claude.com/docs/en/build-with-claude/preserved-thinking#prefix-check) by default, so its system prompt edits were accepted. On accounts created on or after August 31, 2026, 00:00 UTC, a request to one of these four models that sends earlier thinking blocks back gets a 400 error by default if it changes `system`, `tools`, or a message before one of them.

1038 * **What was measured:** The triage agent’s long variant from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run August 24, 2026, on Claude Sonnet 5 with the 5-minute cache, cost from the usage fields at list prices, five sessions per arm: a no-change arm at the default effort throughout ($0.81 USD per session), and two arms that start at low effort and make the same two cache-breaking changes, a switch to the default effort and one added tool, either mid-session at requests 12 and 17 ($0.95 USD) or together on the first request after the first compaction ($0.75 USD).

1039 * **Fourth arm:** A fourth arm of six sessions, run August 25, 2026, made the same two changes on the request that triggered the first compaction ($0.92 USD per session): that request’s summarization pass wrote the 81,000-token context to the cache instead of reading it, so that pass cost $0.21 USD against $0.04 USD for the same pass in the boundary arm.

1040 * **Compaction:** Sessions first compacted at request 21 to 25 (16 of the 21 sessions at request 22), once the prompt passed the 80,000-token compaction trigger, and two no-change sessions compacted a second time near the end. The boundary arm’s lower total than the no-change arm reflects its low-effort requests before the change and those second compactions rather than caching: the two arms’ re-write costs differ by under $0.01 USD.

10411041 * **Mid-session arm:** The mid-session arm paid $0.23 USD per session in cache re-writes; the difference between the mid-session and boundary arms was $0.20 USD with a 95% confidence interval of $0.11 USD to $0.29 USD. In the mid-session arm, the effort change and the added tool rewrote 39,000 and 60,000 cached tokens. One of its sessions ran cheap ($0.82 USD) after its model mis-called the search tool following compaction and got empty results; it is included, and without it the arm averages $0.98 USD.

10421042 * **Accuracy and cache reads:** Accuracy averaged 14.2 of 20 labels in each August 24 arm and 14.7 in the August 25 arm; cache reads were 91% of prompt tokens with no changes, 85% mid-session, 91% at the boundary, and 86% with the changes on the triggering request.

10431043 * **Separate measurement, status line:** A separate measurement on the 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run August 18, 2026, also on Claude Sonnet 5 with the 5-minute cache, priced the same way, three runs per arm: with a 25-token status line at the front of the system prompt, a run cost $4.24 USD on average ($4.12 USD to $4.47 USD across the three runs) and read nothing from the cache; with the same line at the end of the newest user turn, after the last cache breakpoint, a run cost $0.59 USD ($0.55 USD to $0.61 USD). The first is more than the same job cost with caching off. A later re-run with the same configuration gave $4.28 USD and $0.61 USD.

1044 * **Compaction trigger on Claude Haiku 5.5:** Measured October 21, 2026, on the Messages API with [compaction at a token threshold](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold), no tools, adaptive thinking, the default effort (`medium`), the 5-minute cache, a cache breakpoint on the system prompt, and [automatic caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching). Each session was one conversation of 45 requests: a 5,000-token system prompt, 10,000 tokens of filler text with every user turn, and one-word assistant turns, except that a reply with a compaction block was sent back whole. Four settings ran three sessions each: triggers of 150,000 tokens (the default), 90,000, and 50,000 (the lowest allowed), and no compaction. Costs within a setting agreed to within 4%. With the default trigger, 15 of each session's 45 replies and all 3 summaries had prompts over 100,000 tokens. With a trigger of 90,000, the largest prompt held 97,269 tokens. Compaction ran at the first request whose whole prompt had reached the trigger, counting the system prompt, cache reads, and cache writes. The summary was written from that prompt plus 44 tokens, up to 7,269 tokens over the trigger. Three more sessions, with a 22,000-token system prompt and a trigger of 50,000, showed that the system prompt counts. The summaries were short (184 to 640 output tokens) because the text was filler. The requests ran in one Anthropic-internal organization on the Claude API.

1045 * **Compaction trigger on Claude Haiku 5.5, how priced:** From the `usage` fields at list prices, a session cost $0.272 USD with the default trigger, $0.082 USD at 90,000, $0.075 USD at 50,000, and $0.729 USD with no compaction. Per request, from the request after the first compaction through the last compaction, the three triggers cost $0.00605 USD, $0.00181 USD, and $0.00166 USD. With every prompt at the prices for prompts up to 100,000 tokens, the default would cost $0.00206 USD per request, and a trigger of 90,000 would save 12%. A compaction's summary and reply are each priced by their own prompt's size.

1044 * **Compaction trigger on Claude Haiku 5.5:** Measured October 21, 2026, on the Messages API with [compaction at a token threshold](https://platform.claude.com/docs/en/build-with-claude/compaction-threshold), no tools, adaptive thinking, the default effort (`medium`), the 5-minute cache, a cache breakpoint on the system prompt, and [automatic caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching#automatic-caching). Each session was one conversation of 45 requests: a 5,000-token system prompt, 10,000 tokens of filler text with every user turn, and one-word assistant turns, except that a reply with a compaction block was sent back whole. Four settings ran three sessions each: triggers of 150,000 tokens (the default), 90,000, and 50,000 (the lowest allowed), and no compaction. Costs within a setting agreed to within 4%. With the default trigger, 15 of each session’s 45 replies and all 3 summaries had prompts over 100,000 tokens. With a trigger of 90,000, the largest prompt held 97,269 tokens. Compaction ran at the first request whose whole prompt had reached the trigger, counting the system prompt, cache reads, and cache writes. The summary was written from that prompt plus 44 tokens, up to 7,269 tokens over the trigger. Three more sessions, with a 22,000-token system prompt and a trigger of 50,000, showed that the system prompt counts. The summaries were short (184 to 640 output tokens) because the text was filler. The requests ran in one Anthropic-internal organization on the Claude API.

1045 * **Compaction trigger on Claude Haiku 5.5, how priced:** From the `usage` fields at list prices, a session cost $0.272 USD with the default trigger, $0.082 USD at 90,000, $0.075 USD at 50,000, and $0.729 USD with no compaction. Per request, from the request after the first compaction through the last compaction, the three triggers cost $0.00605 USD, $0.00181 USD, and $0.00166 USD. With every prompt at the prices for prompts up to 100,000 tokens, the default would cost $0.00206 USD per request, and a trigger of 90,000 would save 12%. A compaction’s summary and reply are each priced by their own prompt’s size.

10461046

104719. **Cache duration measurement on Claude Fable 5.1:** The same 20-issue triage job and harness as reference 16, with `max_tokens` raised to 4,096, run August 23 and August 26, 2026, on the Claude Fable 5.1 launch snapshot at its launch prices ($10 USD input, $12.50 USD 5-minute write, $20 USD 1-hour write, $0.25 USD cache read, $50 USD output per million tokens), three settings per schedule: the 5-minute cache, the 1-hour cache, and the 5-minute cache kept warm by a `max_tokens: 0` request on the unchanged prefix every 4 minutes, timed from the previous request's start (the August 23 runs sent keep-alive requests with `max_tokens: 1`; in the August 26 cells reported here, every answered keep-alive request refreshed the cache and its response reported no output tokens). Schedules: no pauses, 10% of turns, and every turn at 6 minutes on all 20 issues, and 45-minute pauses on the 5-issue subset; three runs per cell (six for the August 26 keep-alive cell with 45-minute pauses), cost computed from each response's `usage` fields at list prices, accuracy against the same gold labels (12 to 18 exact labels of 20). Per-session means on August 26 for the 5-minute, 1-hour, and keep-alive settings: no pauses $2.42 USD, $3.09 USD, $2.29 USD; 10% paused $4.50 USD, $2.96 USD, $2.36 USD; every turn $22.89 USD, $3.01 USD, $2.62 USD; the August 23 cells agree within 6%. The 45-minute figures ($1.68 USD, $0.59 USD, and $0.71 USD per 5-issue session) are from later sessions of August 26: that day's first set of sessions was cut off when its machine was shut down, and the second was affected by a fault in the service and is set aside. The third set gives the 5-minute and 1-hour figures; its three keep-alive sessions sent each keep-alive request once, lost requests to rate-limit refusals, and are not in the mean. The keep-alive figure is the mean of six further sessions that sent any refused keep-alive request again within two seconds. The August 23 runs gave $1.67 USD, $0.58 USD, and $0.70 USD. The break-even between the 5-minute and 1-hour settings is 3.1% of turns, the same measure as reference 16. Claude Fable 5.1's cache read costs 0.025x the input price while its cache writes keep the standard multipliers, so a keep-alive request that re-reads the prefix is cheap and the 1-hour duration's write premium is the larger bill, except with 45-minute pauses, where the keep-alive requests cost more than the premium.

104719. **Cache duration measurement on Claude Fable 5.1:** The same 20-issue triage job and harness as reference 16, with `max_tokens` raised to 4,096, run August 23 and August 26, 2026, on the Claude Fable 5.1 launch snapshot at its launch prices ($10 USD input, $12.50 USD 5-minute write, $20 USD 1-hour write, $0.25 USD cache read, $50 USD output per million tokens), three settings per schedule: the 5-minute cache, the 1-hour cache, and the 5-minute cache kept warm by a `max_tokens: 0` request on the unchanged prefix every 4 minutes, timed from the previous request’s start (the August 23 runs sent keep-alive requests with `max_tokens: 1`; in the August 26 cells reported here, every answered keep-alive request refreshed the cache and its response reported no output tokens). Schedules: no pauses, 10% of turns, and every turn at 6 minutes on all 20 issues, and 45-minute pauses on the 5-issue subset; three runs per cell (six for the August 26 keep-alive cell with 45-minute pauses), cost computed from each response’s `usage` fields at list prices, accuracy against the same gold labels (12 to 18 exact labels of 20). Per-session means on August 26 for the 5-minute, 1-hour, and keep-alive settings: no pauses $2.42 USD, $3.09 USD, $2.29 USD; 10% paused $4.50 USD, $2.96 USD, $2.36 USD; every turn $22.89 USD, $3.01 USD, $2.62 USD; the August 23 cells agree within 6%. The 45-minute figures ($1.68 USD, $0.59 USD, and $0.71 USD per 5-issue session) are from later sessions of August 26: that day’s first set of sessions was cut off when its machine was shut down, and the second was affected by a fault in the service and is set aside. The third set gives the 5-minute and 1-hour figures; its three keep-alive sessions sent each keep-alive request once, lost requests to rate-limit refusals, and are not in the mean. The keep-alive figure is the mean of six further sessions that sent any refused keep-alive request again within two seconds. The August 23 runs gave $1.67 USD, $0.58 USD, and $0.70 USD. The break-even between the 5-minute and 1-hour settings is 3.1% of turns, the same measure as reference 16. Claude Fable 5.1’s cache read costs 0.025x the input price while its cache writes keep the standard multipliers, so a keep-alive request that re-reads the prefix is cheap and the 1-hour duration’s write premium is the larger bill, except with 45-minute pauses, where the keep-alive requests cost more than the premium.

10481048

104920. **Terminal-Bench 3:** the public terminal-agent benchmark's 74 tasks, run on [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/overview) with two custom tools, a shell and a file editor that the evaluation harness runs in each task's own container, in place of the platform's built-in tools, and otherwise at the platform's default settings for external accounts, two runs per model at `high` effort, August 27 to 28, 2026. These runs used Terminal-Bench version 3.0, and their scores are not comparable with the public Terminal-Bench leaderboard or with the Terminal-Bench 4.0 results in the Claude Opus 5.5 system card, which come from runs in Claude Code at `max` effort. Each task's time limits are 2.5 times the benchmark's own, which gives the agent between 75 minutes and 20 hours per task (5 hours for the median task), and each task gets three times the memory it specifies, from 6 GiB to 96 GiB, with extra memory for the 12 tasks that run helper services. The agent had no general internet access: its containers could reach an internal package mirror, a short list of download sites including GitHub and the Python Package Index, and a few sites specific to some tasks, and eight of the tasks had no network access at all. Scores are raw pass rates over the 148 attempts per model; single runs swing by 5 to 11 points. Costs are what a customer would be billed at list prices, re-priced request by request from the runs' usage records with the 5-minute cache lifetime. Claude Opus 4.7, Claude Opus 4.8, and Claude Opus 5 each spend $8 USD to $15 USD per task. Claude Opus 4.7 ended 11 of its 148 attempts at its output cap.

104920. **Terminal-Bench 3:** the public terminal-agent benchmark’s 74 tasks, run on [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/overview) with two custom tools, a shell and a file editor that the evaluation harness runs in each task’s own container, in place of the platform’s built-in tools, and otherwise at the platform’s default settings for external accounts, two runs per model at `high` effort, August 27 to 28, 2026. These runs used Terminal-Bench version 3.0, and their scores are not comparable with the public Terminal-Bench leaderboard or with the Terminal-Bench 4.0 results in the Claude Opus 5.5 system card, which come from runs in Claude Code at `max` effort. Each task’s time limits are 2.5 times the benchmark’s own, which gives the agent between 75 minutes and 20 hours per task (5 hours for the median task), and each task gets three times the memory it specifies, from 6 GiB to 96 GiB, with extra memory for the 12 tasks that run helper services. The agent had no general internet access: its containers could reach an internal package mirror, a short list of download sites including GitHub and the Python Package Index, and a few sites specific to some tasks, and eight of the tasks had no network access at all. Scores are raw pass rates over the 148 attempts per model; single runs swing by 5 to 11 points. Costs are what a customer would be billed at list prices, re-priced request by request from the runs’ usage records with the 5-minute cache lifetime. Claude Opus 4.7, Claude Opus 4.8, and Claude Opus 5 each spend $8 USD to $15 USD per task. Claude Opus 4.7 ended 11 of its 148 attempts at its output cap.

10501050

105121. **DRACO:** Perplexity, "DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity," arXiv:2602.11685, 2026.

105121. **DRACO:** Perplexity, “DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity,” arXiv:2602.11685, 2026.

10521052

10531053 * **What was measured:** Its 100 research tasks across 10 domains are graded against expert-written rubrics of about 40 weighted criteria each, and the score is the weighted share of criteria met, from 0 to 100.

1054 * **Configurations:** Every configuration ran on the Claude API with Claude Opus 5.5, the default adaptive thinking, the production safety classifiers on, and `max_tokens` at 128,000 with streaming: a single agent at `medium` effort (the model's default) and at `low`, the single agent at `medium` with the sentence and the clock, and a team at `medium` with and without them. The team is a lead agent that can start helper agents of the same model through a tool, with no cap on their number.

1055 * **The sentence:** The two changes are a sentence and a clock. The sentence, "Time matters here: do not spend time that can be avoided, and the earlier a correct result is obtained, the better.", goes at the end of the task in the agent's first `user` message, after a blank line, and at the end of each helper's brief.

1056 * **The clock:** The clock is the elapsed time in whole seconds, such as `[elapsed 252s]`, added before every request, including the first, to the end of the newest `user` message: in its own text block after the task on the first request, and at the end of the last tool result's text, after a blank line, when the message carries tool results. Earlier clocks stay in place, so the cached prefix still matches on the next request. Every agent in a team reads the same clock, and the system prompt doesn't change.

1057 * **In the measured runs:** The measured runs used this wording and format. Their clock started at the first token of the first response rather than at the start of the task. In the measured team runs, a helper's report reached the lead in a separate message after the clock, not inside the last tool result.

1054 * **Configurations:** Every configuration ran on the Claude API with Claude Opus 5.5, the default adaptive thinking, the production safety classifiers on, and `max_tokens` at 128,000 with streaming: a single agent at `medium` effort (the model’s default) and at `low`, the single agent at `medium` with the sentence and the clock, and a team at `medium` with and without them. The team is a lead agent that can start helper agents of the same model through a tool, with no cap on their number.

1055 * **The sentence:** The two changes are a sentence and a clock. The sentence, “Time matters here: do not spend time that can be avoided, and the earlier a correct result is obtained, the better.”, goes at the end of the task in the agent’s first `user` message, after a blank line, and at the end of each helper’s brief.

1056 * **The clock:** The clock is the elapsed time in whole seconds, such as `[elapsed 252s]`, added before every request, including the first, to the end of the newest `user` message: in its own text block after the task on the first request, and at the end of the last tool result’s text, after a blank line, when the message carries tool results. Earlier clocks stay in place, so the cached prefix still matches on the next request. Every agent in a team reads the same clock, and the system prompt doesn’t change.

1057 * **In the measured runs:** The measured runs used this wording and format. Their clock started at the first token of the first response rather than at the start of the task. In the measured team runs, a helper’s report reached the lead in a separate message after the clock, not inside the last tool result.

10581058 * **When:** Each configuration made three attempts at each task, with a four-hour limit per attempt, run September 24, 2026, and all configurations of a set started together.

10591059 * **Tools:** The agents used search and fetch tools that the evaluation harness hosts over an Anthropic web index, and Python and a shell in a sandbox with no network access. Those tools set part of the time, and yours will run at a different speed, so the page gives time as a ratio between configurations, not in minutes.

1060 * **How timed:** Time is the wall-clock time of an attempt, from its first request to the end of the last request by any agent, including tool runs, averaged over a task's attempts. A typical-task figure is the median, over tasks, of each task's ratio between two configurations.

1061 * **How priced:** Each task's cost is its requests priced at Claude Opus 5.5's public list prices, with prompt caching billed as it would be for a customer who sets a cache breakpoint at the end of each request and uses the 5-minute cache lifetime. Each request reads the previous request's prompt from the cache and writes the rest. An attempt's first request, and any request that starts more than 5 minutes after the one before it, writes its whole prompt. Costs cover model tokens only; the harness's tools add no charges.

1062 * **How scored:** All intervals are 95% bootstrap intervals over tasks, and score changes are paired differences over tasks. Claude Opus 5 grades the answers, so Claude Opus 5.5 doesn't grade its own work, and each answer's score averages five judging runs.

1063 * **Caveats:** A production safety classifier stopped three attempts in each configuration, all on one task, and those attempts score 0; leaving that task out changes no finding. The runs also passed each agent's tool use through an Anthropic-internal safety check that customer requests don't have, which adds time to each request in every configuration. On DRACO, the single agent without the changes made about twice as many requests, and the team about a third more, so the check probably makes the DRACO time savings look slightly larger.

1060 * **How timed:** Time is the wall-clock time of an attempt, from its first request to the end of the last request by any agent, including tool runs, averaged over a task’s attempts. A typical-task figure is the median, over tasks, of each task’s ratio between two configurations.

1061 * **How priced:** Each task’s cost is its requests priced at Claude Opus 5.5’s public list prices, with prompt caching billed as it would be for a customer who sets a cache breakpoint at the end of each request and uses the 5-minute cache lifetime. Each request reads the previous request’s prompt from the cache and writes the rest. An attempt’s first request, and any request that starts more than 5 minutes after the one before it, writes its whole prompt. Costs cover model tokens only; the harness’s tools add no charges.

1062 * **How scored:** All intervals are 95% bootstrap intervals over tasks, and score changes are paired differences over tasks. Claude Opus 5 grades the answers, so Claude Opus 5.5 doesn’t grade its own work, and each answer’s score averages five judging runs.

1063 * **Caveats:** A production safety classifier stopped three attempts in each configuration, all on one task, and those attempts score 0; leaving that task out changes no finding. The runs also passed each agent’s tool use through an Anthropic-internal safety check that customer requests don’t have, which adds time to each request in every configuration. On DRACO, the single agent without the changes made about twice as many requests, and the team about a third more, so the check probably makes the DRACO time savings look slightly larger.

10641064 * **Results, single agent:** Results for [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time): with the changes, the single agent took 47% less time at a 60% lower cost on the typical task, its average cost fell from $1.13 USD to $0.44 USD per task, and its score was 4.1 points lower (95% interval 2.9 to 5.2 lower).

1065 * **Results, lower effort:** Lowering the single agent's effort saved more time and cost than the changes did, and lost more score. At `low` effort, it scored 13.3 points lower on DRACO, 3.4 on HLE, and 14.0 on the physics set than at `medium`, and the typical task took 84%, 30%, and 47% less time. With the changes at `medium` effort, it scored 9.2, 2.5, and 11.0 points higher than at `low`, and the typical task took longer than at `low` by a factor of 1.2 to 3.2.

1066 * **Results, team, helpers:** The team's lead rarely started helpers. Without the changes, it started a helper in only 5 of 300 attempts on DRACO, 13 of 1,479 on HLE, and none of 280 on the physics set.

1067 * **Results, team:** So the team results mostly show the lead agent working alone, with the team's own instructions and tools. Without the changes, the team already took about half the single agent's time on the typical DRACO task, and scored 3.3 points lower (95% interval 2.1 to 4.6 lower). With the changes, the team took 17% less time at a 20% lower cost on the typical DRACO task, and scored 1.6 points lower (95% interval 0.3 to 2.8 lower).

1065 * **Results, lower effort:** Lowering the single agent’s effort saved more time and cost than the changes did, and lost more score. At `low` effort, it scored 13.3 points lower on DRACO, 3.4 on HLE, and 14.0 on the physics set than at `medium`, and the typical task took 84%, 30%, and 47% less time. With the changes at `medium` effort, it scored 9.2, 2.5, and 11.0 points higher than at `low`, and the typical task took longer than at `low` by a factor of 1.2 to 3.2.

1066 * **Results, team, helpers:** The team’s lead rarely started helpers. Without the changes, it started a helper in only 5 of 300 attempts on DRACO, 13 of 1,479 on HLE, and none of 280 on the physics set.

1067 * **Results, team:** So the team results mostly show the lead agent working alone, with the team’s own instructions and tools. Without the changes, the team already took about half the single agent’s time on the typical DRACO task, and scored 3.3 points lower (95% interval 2.1 to 4.6 lower). With the changes, the team took 17% less time at a 20% lower cost on the typical DRACO task, and scored 1.6 points lower (95% interval 0.3 to 2.8 lower).

10681068

106922. **HLE:** Phan et al., "Humanity's Last Exam," arXiv:2501.14249, 2025. Expert-written questions with exact answers, graded against the reference answers. Measured on the first 500 questions of the September 10, 2025, release, images included, with the benchmark's own sources blocked from search, and the same setup as reference 21. Each configuration made three attempts at each question, run September 24, 2026. Claude Opus 5 compares each answer with the reference answer, using the benchmark's published judge prompt, and the majority of three judgings counts. The scores cover 493 questions in every configuration. For five of the others, the grading request was over the judge's 1M-token limit, and the other two are left out because the run of the single agent without the changes stopped before three of its attempts on them finished. Time and cost ratios cover 492 questions for the single agent and 491 for the team, because one or two questions have no timing. A production safety classifier stopped 3% to 4% of each configuration's attempts, and those attempts score 0; leaving them out changes no finding. Results for [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time): with the changes, the single agent took 14% less time at a 9% lower cost on the typical question. With the Claude Opus 5 judge, its score showed no clear change (0.9 points lower, 95% interval 2.5 lower to 0.7 higher). With a second grader, Claude Opus 4.6, its score was 1.9 points lower (95% interval 0.2 to 3.7 lower). Average time fell further than the typical question's: average time fell 43% and average cost 37%, because the changes cut a few very long questions short. The team's time on the typical question changed by 3% or less, and its score didn't change clearly.

106922. **HLE:** Phan et al., “Humanity’s Last Exam,” arXiv:2501.14249, 2025. Expert-written questions with exact answers, graded against the reference answers. Measured on the first 500 questions of the September 10, 2025, release, images included, with the benchmark’s own sources blocked from search, and the same setup as reference 21. Each configuration made three attempts at each question, run September 24, 2026. Claude Opus 5 compares each answer with the reference answer, using the benchmark’s published judge prompt, and the majority of three judgings counts. The scores cover 493 questions in every configuration. For five of the others, the grading request was over the judge’s 1M-token limit, and the other two are left out because the run of the single agent without the changes stopped before three of its attempts on them finished. Time and cost ratios cover 492 questions for the single agent and 491 for the team, because one or two questions have no timing. A production safety classifier stopped 3% to 4% of each configuration’s attempts, and those attempts score 0; leaving them out changes no finding. Results for [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time): with the changes, the single agent took 14% less time at a 9% lower cost on the typical question. With the Claude Opus 5 judge, its score showed no clear change (0.9 points lower, 95% interval 2.5 lower to 0.7 higher). With a second grader, Claude Opus 4.6, its score was 1.9 points lower (95% interval 0.2 to 3.7 lower). Average time fell further than the typical question’s: average time fell 43% and average cost 37%, because the changes cut a few very long questions short. The team’s time on the typical question changed by 3% or less, and its score didn’t change clearly.

10701070

107123. **Physics set:** An internal set of 70 research-level physics problems, adapted from the public CritPt benchmark: Zhu et al., "Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark," arXiv:2509.26574, 2025. Expert reviewers corrected the problem statements. Claude Opus 4.6 grades each answer with partial credit, using the set's grading prompt, against expert reference solutions that aren't public, so the scores can't be compared with published results. The score is the mean grade over a problem's attempts, averaged over problems, in points out of 100. Measured on all 70 problems, four attempts per problem, run September 24, 2026. Every agent had a Python tool, a shell, and a file editor in a sandbox container with no network access, and no search or fetch tools. No safety classifier stopped an attempt on the physics set. Otherwise the setup is that of reference 21. Results for [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time): with the changes, the single agent took 7% less time at a 4% lower cost on the typical problem, and its score was 3.0 points lower (95% interval 0.4 to 5.6 lower). The team's time on the typical problem changed by 3% or less, and its score didn't change clearly. The single agent's average time fell 14%. The single agent's whole average time saving comes from one problem, on which the agent without the changes ran code for about half an hour per attempt; without that problem, its average time is about the same with and without the changes.

107123. **Physics set:** An internal set of 70 research-level physics problems, adapted from the public CritPt benchmark: Zhu et al., “Probing the Critical Point (CritPt) of AI Reasoning: a Frontier Physics Research Benchmark,” arXiv:2509.26574, 2025. Expert reviewers corrected the problem statements. Claude Opus 4.6 grades each answer with partial credit, using the set’s grading prompt, against expert reference solutions that aren’t public, so the scores can’t be compared with published results. The score is the mean grade over a problem’s attempts, averaged over problems, in points out of 100. Measured on all 70 problems, four attempts per problem, run September 24, 2026. Every agent had a Python tool, a shell, and a file editor in a sandbox container with no network access, and no search or fetch tools. No safety classifier stopped an attempt on the physics set. Otherwise the setup is that of reference 21. Results for [Show the model elapsed time](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#show-the-model-elapsed-time): with the changes, the single agent took 7% less time at a 4% lower cost on the typical problem, and its score was 3.0 points lower (95% interval 0.4 to 5.6 lower). The team’s time on the typical problem changed by 3% or less, and its score didn’t change clearly. The single agent’s average time fell 14%. The single agent’s whole average time saving comes from one problem, on which the agent without the changes ran code for about half an hour per attempt; without that problem, its average time is about the same with and without the changes.

10721072

1073107324. **tau2-bench retail:** Barres et al., "τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment," arXiv:2506.07982, 2025.

10741074

1075 * **What was measured:** The retail domain's 114 tasks ran on the public harness against the Claude API, with Claude Haiku 4.5 as the simulated user, Claude Sonnet 5 as the natural-language judge, and prompt caching on; the tasks were split into 57 search and 57 held-out.

1076 * **When:** The two searches ran on September 6, 2026, driven by the Claude API skill's `hillclimb` command. The caching before-and-after pair ran on September 8, 2026.

1077 * **How scored:** In the searches, the starting points and the confirmed configuration are the mean of three held-out runs, kept steps were judged on two runs, and other candidates were scored on one or two runs. The caching pair: three runs each on all 114 tasks with only the harness's cache breakpoints toggled, the configuration the `cost-optimize` command produced in an earlier run. Held-out numbers are on 57 tasks the search never read.

1078 * **How priced:** Cost per task prices the agent's and the simulated user's token counts at list prices (Claude Opus 5 $5 USD and $25 USD, Claude Sonnet 5 $2 USD and $10 USD per million input and output tokens, cache reads and writes at their published rates) and excludes the judge; percent changes come from unrounded costs; the monthly figure is the agent's tokens alone, $0.135 USD and $0.047 USD per task, times 10,000. At 10,000 conversations a month, that is about $1,350 USD before the first search and $470 USD after.

1075 * **What was measured:** The retail domain’s 114 tasks ran on the public harness against the Claude API, with Claude Haiku 4.5 as the simulated user, Claude Sonnet 5 as the natural-language judge, and prompt caching on; the tasks were split into 57 search and 57 held-out.

1076 * **When:** The two searches ran on September 6, 2026, driven by the Claude API skill’s `hillclimb` command. The caching before-and-after pair ran on September 8, 2026.

1077 * **How scored:** In the searches, the starting points and the confirmed configuration are the mean of three held-out runs, kept steps were judged on two runs, and other candidates were scored on one or two runs. The caching pair: three runs each on all 114 tasks with only the harness’s cache breakpoints toggled, the configuration the `cost-optimize` command produced in an earlier run. Held-out numbers are on 57 tasks the search never read.

1078 * **How priced:** Cost per task prices the agent’s and the simulated user’s token counts at list prices (Claude Opus 5 $5 USD and $25 USD, Claude Sonnet 5 $2 USD and $10 USD per million input and output tokens, cache reads and writes at their published rates) and excludes the judge; percent changes come from unrounded costs; the monthly figure is the agent’s tokens alone, $0.135 USD and $0.047 USD per task, times 10,000. At 10,000 conversations a month, that is about $1,350 USD before the first search and $470 USD after.

10791079 * **Caveats:** These measurements ran on the previous Claude Opus and Claude Sonnet models: the caching pair ran on Claude Sonnet 5, the second search started from Claude Sonnet 5, and the first search went from Claude Opus 5 to Claude Sonnet 5.

10801080 * **Caching pair:** The pair ran on Claude Sonnet 5 at the default effort. The public harness, as shipped, sends the whole conversation uncached on every turn. The cached configuration has one breakpoint on the system prompt and one on the latest message. Both searches ran with that caching on. Cost per task went from $0.223 USD uncached to $0.062 USD cached, a 72% saving, and the pass rate from 79.2% to 76.0%. Identical cached runs of that configuration ranged from 85 to 93 of 114 across days. Every cached run scored at or below every uncached run, but the gap of under 4 tasks sits inside the spread of identical cached runs, so the accuracy change is too small to measure at this sample size.

10811081 * **How the search ran.** The `hillclimb` command asks four questions, presents a plan, and waits for approval before it spends anything. In the first search, the plan left the user simulator, grader, caching, temperature, and tasks unchanged. The last of the four questions began: `One pass over the full set is 114 cases x 1 rep on claude-opus-5, roughly 22 minutes.` Each round, the search proposes one change and runs the full set. It keeps the change only when the number moved and the predicted mechanism shows in the usage data. It stops when three rounds in a row bring nothing, then confirms the winner on fresh runs.

10821082 * **First search, result.** The search cut cost per task by 61% while holding the pass rate within a 2-point band set before the search: from Claude Opus 5 at the default effort ($0.142 USD a task, 76.0% passed) to Claude Sonnet 5 at `low` with one cost-neutral prompt edit ($0.055 USD, 74.3%). The search cost $179 USD in evaluation runs and $228 USD in its own tokens.

10831083 * **First search, start: Claude Opus 5, default effort.** 76.0% at $0.142 USD. Three identical runs spanned 43 to 44 of 57. The search set the floor 2 points under, at 74.0%, and rounded it up to 43 of 57 (75.4%) for its verdicts.

1084 * **First search, step 1, effort to `low`: kept.** 77.2% on the two runs that kept it, 77.8% on a three-run confirm after the search, at $0.119 USD, 17% cheaper. The search's own status line reads, in part: `two runs held-out 42 and 46, mean 44.0 >= floor 43; cost mean $0.1187 <= gate $0.1208; mechanism confirmed (output tokens -34% with calls flat)`.

1084 * **First search, step 1, effort to `low`: kept.** 77.2% on the two runs that kept it, 77.8% on a three-run confirm after the search, at $0.119 USD, 17% cheaper. The search’s own status line reads, in part: `two runs held-out 42 and 46, mean 44.0 >= floor 43; cost mean $0.1187 <= gate $0.1208; mechanism confirmed (output tokens -34% with calls flat)`.

10851085 * **First search, step 2, Claude Sonnet 5 at `low`: kept.** 75.4% at $0.055 USD, 61% cheaper.

1086 * **First search, step 3, a prompt edit (look the customer's orders up after login instead of asking for an order ID): kept on selection, then did not hold.** 79.8% on two runs; 74.3% on three fresh runs. The search left it in as cost-neutral: the freed turn became another lookup turn.

1087 * **First search, step 4, a prompt edit (transfer to a human only when asked): rejected and reverted.** 71.9%. It fixed the 3 cases it targeted and broke 10 others with a generic "say what you can and can't do" template.

1088 * **First search, step 5, a prompt edit: not run.** Its predicted gain of 0.5 tasks was under the search's 2-task adoption gate.

1089 * **First search, confirm: Claude Sonnet 5 at `low` with the step-3 prompt, three fresh runs.** 74.3% at $0.055 USD, inside the 2-point band by 0.14 of a task, 1 task in 57 under the start, and below the 43-of-57 floor the search registered. This configuration's five runs spanned 42 to 46, wider than the start's 43 to 44.

1090 * **First search, what it showed.** The effort and model levers produced the savings, and each one's mechanism confirmed it. The prompt lever produced real behavior changes but no durable gain. On this workload, the number of turns and the cached prefix each turn re-reads set the cost per task, not the wording.

1091 * **Second search.** It started from Claude Sonnet 5 at the default effort with caching on: 79.5% at $0.062 USD on the held-out half, with three runs spanning 42 to 47 of 57, so the floor was 45 of 57 (78.9%). Identical runs of a starting configuration differed by up to 5 tasks in 57 here. The second search kept nothing. Effort at `low`, two prompt edits, and both together each scored 77.2%, under the floor, and saved 10% to 14%, short of the search's 15% cost gate. A prompt edit for accuracy scored 79.8% on two runs and 73.7% on three fresh runs, and was reverted. The search took about 19 evaluation runs and $120 USD.

1086 * **First search, step 3, a prompt edit (look the customer’s orders up after login instead of asking for an order ID): kept on selection, then did not hold.** 79.8% on two runs; 74.3% on three fresh runs. The search left it in as cost-neutral: the freed turn became another lookup turn.

1088 * **First search, step 5, a prompt edit: not run.** Its predicted gain of 0.5 tasks was under the search’s 2-task adoption gate.

1089 * **First search, confirm: Claude Sonnet 5 at `low` with the step-3 prompt, three fresh runs.** 74.3% at $0.055 USD, inside the 2-point band by 0.14 of a task, 1 task in 57 under the start, and below the 43-of-57 floor the search registered. This configuration’s five runs spanned 42 to 46, wider than the start’s 43 to 44.

1090 * **First search, what it showed.** The effort and model levers produced the savings, and each one’s mechanism confirmed it. The prompt lever produced real behavior changes but no durable gain. On this workload, the number of turns and the cached prefix each turn re-reads set the cost per task, not the wording.

1091 * **Second search.** It started from Claude Sonnet 5 at the default effort with caching on: 79.5% at $0.062 USD on the held-out half, with three runs spanning 42 to 47 of 57, so the floor was 45 of 57 (78.9%). Identical runs of a starting configuration differed by up to 5 tasks in 57 here. The second search kept nothing. Effort at `low`, two prompt edits, and both together each scored 77.2%, under the floor, and saved 10% to 14%, short of the search’s 15% cost gate. A prompt edit for accuracy scored 79.8% on two runs and 73.7% on three fresh runs, and was reverted. The search took about 19 evaluation runs and $120 USD.

10921092

1093109325. **Claude Managed Agents task set:**

10941094

10951095 * **What was measured:** An Anthropic-internal set of 79 scripted agent tasks in eight areas, including memory, multi-agent, and long-context work, run as [Claude Managed Agents](https://platform.claude.com/docs/en/managed-agents/overview) sessions in which the test answers every tool call. Each task ran five times per model.

10961096 * **Runs:** The figures are from one run per model on October 7, 2026, graded that day. In three more runs of the same tasks on October 8 and 9, 2026, Claude Haiku 5.5 passed 91% to 93% of attempts, and in one more run on October 8, Claude Haiku 4.5 passed 63%.

1097 * **How scored:** An attempt passed only if the grader, Claude Sonnet 4.6, grading once, marked every line of the task's checklist as met, so pass rates depend on the grader. On all but one task, a separate call, shown the tool results in full, graded the line that checks the files and messages the agent wrote through its tools against those results.

1097 * **How scored:** An attempt passed only if the grader, Claude Sonnet 4.6, grading once, marked every line of the task’s checklist as met, so pass rates depend on the grader. On all but one task, a separate call, shown the tool results in full, graded the line that checks the files and messages the agent wrote through its tools against those results.

10981098 * **How priced:** Costs are token costs at list prices with 5-minute prompt caching, priced request by request, and exclude session runtime.

1099 * **Results, token prices and cost:** Claude Haiku 4.5 did not cache 80% of its requests: each had a prompt of about 4,100 tokens or fewer, and its minimum cacheable prompt length is 4,096 tokens (512 for Claude Haiku 5.5). From Claude Haiku 4.5 to Claude Haiku 5.5, token prices fell by up to 90%, and the token cost of the 79 tasks fell by about 75%. The token cost fell less than the prices because about 5% of Claude Haiku 5.5's requests had prompts over 100,000 tokens, which pay higher prices ([Long context pricing](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing)), and they carried 73% of its cost.

1100 * **Thinking:** Anthropic set Claude Haiku 5.5's effort to `medium` and sent no thinking setting or output limit for either model, so thinking was left to Claude Managed Agents' own settings for each model. Claude Haiku 5.5 used thinking in 59% of requests, and Claude Haiku 4.5 in 22%. The comparison is of each model as served, not at equal settings.

1101 * **Cost without caching, priced from recorded token counts:** Anthropic also re-priced the recorded token counts of Claude Haiku 5.5's October 7 run for the caching factor in [Why caching comes first](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#why-caching-comes-first). Claude Managed Agents applies prompt caching itself, so the factor compares two prices for the same requests, not two settings.

1102 * **The factor:** The factor, 1.7, is the run's token cost with every input token at the input price, output tokens unchanged, divided by its token cost with each session priced by itself: each request's prior prompt as a cache read and its new tokens as a 5-minute cache write. Each request is priced for its own prompt length in both costs, so the 87 requests over 100,000 tokens pay the higher prices in both. On the token counts as recorded, which this guide's costs for the run use, the factor is 1.9, because the five attempts of a task ran side by side and read each other's cache entries.

1099 * **Results, token prices and cost:** Claude Haiku 4.5 did not cache 80% of its requests: each had a prompt of about 4,100 tokens or fewer, and its minimum cacheable prompt length is 4,096 tokens (512 for Claude Haiku 5.5). From Claude Haiku 4.5 to Claude Haiku 5.5, token prices fell by up to 90%, and the token cost of the 79 tasks fell by about 75%. The token cost fell less than the prices because about 5% of Claude Haiku 5.5’s requests had prompts over 100,000 tokens, which pay higher prices ([Long context pricing](https://platform.claude.com/docs/en/about-claude/pricing#long-context-pricing)), and they carried 73% of its cost.

1100 * **Thinking:** Anthropic set Claude Haiku 5.5’s effort to `medium` and sent no thinking setting or output limit for either model, so thinking was left to Claude Managed Agents’ own settings for each model. Claude Haiku 5.5 used thinking in 59% of requests, and Claude Haiku 4.5 in 22%. The comparison is of each model as served, not at equal settings.

1101 * **Cost without caching, priced from recorded token counts:** Anthropic also re-priced the recorded token counts of Claude Haiku 5.5’s October 7 run for the caching factor in [Why caching comes first](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#why-caching-comes-first). Claude Managed Agents applies prompt caching itself, so the factor compares two prices for the same requests, not two settings.

1102 * **The factor:** The factor, 1.7, is the run’s token cost with every input token at the input price, output tokens unchanged, divided by its token cost with each session priced by itself: each request’s prior prompt as a cache read and its new tokens as a 5-minute cache write. Each request is priced for its own prompt length in both costs, so the 87 requests over 100,000 tokens pay the higher prices in both. On the token counts as recorded, which this guide’s costs for the run use, the factor is 1.9, because the five attempts of a task ran side by side and read each other’s cache entries.

11031103 * **Why the factor is low:** The factor is low because the sessions are short, a median of 4 requests: most input tokens arrive as tool results on the second or third request and are read from the cache only once or twice. Eight long-context tasks hold three quarters of the cost. A second run, on October 8, also gives 1.7.

11041104

1105110526. **Answer-format measurement:**

11061106

1107 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run on Claude Sonnet 5 with prompt caching on and no thinking setting in the request, so thinking was left at the model's default, under three final-answer instructions, three runs per instruction.

1107 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run on Claude Sonnet 5 with prompt caching on and no thinking setting in the request, so thinking was left at the model’s default, under three final-answer instructions, three runs per instruction.

11081108 * **Two-line instruction:** The original asked for two lines: `4. Finish with exactly two lines: LABEL: <one of: bug-confirmed, needs-more-info, duplicate-candidate, feature-request, upstream-issue, perf, ui-polish> SUMMARY: <one or two sentences for the engineering team>`

11091109 * **One-line instruction:** The shorter variant asked for one: `4. Finish with exactly one line in this form: DECISION | LABEL | REASON where DECISION is one of: triage-now, needs-info, close-duplicate; LABEL is one of: bug-confirmed, needs-more-info, duplicate-candidate, feature-request, upstream-issue, perf, ui-polish; REASON is one clause under 15 words. Output nothing after that line.`

11101110 * **Memo instruction:** The longer variant asked for a memo with six headed sections: problem summary, evidence, duplicate check, recommended label, next steps, and confidence.

1111 * **Also in the instructions as sent:** As sent, the two-line instruction also said "Be concise." and the memo instruction "Be thorough and explicit."

1111 * **Also in the instructions as sent:** As sent, the two-line instruction also said “Be concise.” and the memo instruction “Be thorough and explicit.”

11121112 * **When:** The two-line and one-line instructions ran on August 18, 2026, with `max_tokens` at 700, and the memo instruction on August 19, 2026, with `max_tokens` at 3,000. No response reached its cap.

1113 * **How scored and priced:** Figures are the mean of the three runs at Claude Sonnet 5's list prices, and accuracy is exact match against the gold labels.

1114 * **Results:** The one-line answer used 39% fewer output tokens than the two-line original and cost 13% less per run (from the unrounded means; the chart's rounded costs, $0.49 USD and $0.57 USD, give 14%). The memo used six times the output tokens of the two-line original and cost 2.8 times the one-line answer.

1113 * **How scored and priced:** Figures are the mean of the three runs at Claude Sonnet 5’s list prices, and accuracy is exact match against the gold labels.

1114 * **Results:** The one-line answer used 39% fewer output tokens than the two-line original and cost 13% less per run (from the unrounded means; the chart’s rounded costs, $0.49 USD and $0.57 USD, give 14%). The memo used six times the output tokens of the two-line original and cost 2.8 times the one-line answer.

11151115 * **Thinking:** About half of the responses included thinking, and its tokens are counted in the output tokens.

11161116

1117111727. **Tool search measurement:**

11181118

1119 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run on Claude Sonnet 5 on August 19, 2026, with prompt caching on and no thinking setting in the request, so thinking was left at the model's default, three runs per configuration. The agent had its own two tools plus real tool definitions from public MCP servers, for a total of 7, 17, 52, 102, 252, or 502 tools, with every definition loaded or with the added ones marked `defer_loading` behind tool search.

1120 * **How scored and priced:** Figures are the mean of the three runs at Claude Sonnet 5's list prices, and accuracy is exact match against the gold labels.

1121 * **Results:** With every definition loaded, the run cost rose from $0.55 USD at 7 tools to $1.02 USD at 502 tools, nearly double. With tool search it was $0.55 USD to $0.56 USD at every catalog size, 45% less at 502 tools (from the chart's rounded costs, $0.56 USD and $1.02 USD; the unrounded means give 45.6%). Every run answered 15 to 18 of the 20 issues correctly, with or without tool search.

1122 * **Caveats:** No run called a tool from the catalog or the tool search tool: the agent's own two tools were always loaded, and every tool call went to one of them. The measurement shows what carrying unused definitions costs.

1123 * **Separate measurement:** In a separate measurement on the same model and date, with a public GitHub MCP server attached through the MCP connector and no catalog, three runs with the server's toolset deferred cost 20% less than three runs without ($0.56 USD and $0.70 USD per run), and the three runs of each arm answered 17, 16, and 16 issues correctly. The requests in this separate measurement included no tool search tool. With the toolset loaded, each run called one of the server's tools, `issue_read`, 3 or 4 times; with it deferred, no run called any of the server's tools.

1119 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens), run on Claude Sonnet 5 on August 19, 2026, with prompt caching on and no thinking setting in the request, so thinking was left at the model’s default, three runs per configuration. The agent had its own two tools plus real tool definitions from public MCP servers, for a total of 7, 17, 52, 102, 252, or 502 tools, with every definition loaded or with the added ones marked `defer_loading` behind tool search.

1120 * **How scored and priced:** Figures are the mean of the three runs at Claude Sonnet 5’s list prices, and accuracy is exact match against the gold labels.

1121 * **Results:** With every definition loaded, the run cost rose from $0.55 USD at 7 tools to $1.02 USD at 502 tools, nearly double. With tool search it was $0.55 USD to $0.56 USD at every catalog size, 45% less at 502 tools (from the chart’s rounded costs, $0.56 USD and $1.02 USD; the unrounded means give 45.6%). Every run answered 15 to 18 of the 20 issues correctly, with or without tool search.

1122 * **Caveats:** No run called a tool from the catalog or the tool search tool: the agent’s own two tools were always loaded, and every tool call went to one of them. The measurement shows what carrying unused definitions costs.

1123 * **Separate measurement:** In a separate measurement on the same model and date, with a public GitHub MCP server attached through the MCP connector and no catalog, three runs with the server’s toolset deferred cost 20% less than three runs without ($0.56 USD and $0.70 USD per run), and the three runs of each arm answered 17, 16, and 16 issues correctly. The requests in this separate measurement included no tool search tool. With the toolset loaded, each run called one of the server’s tools, `issue_read`, 3 or 4 times; with it deferred, no run called any of the server’s tools.

11241124

1125112528. **Input trimming and context lifecycle measurements:**

11261126

1127 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) and its long variant, run on Claude Sonnet 5 on August 19, 2026 (dated in UTC), three runs per setting, with no thinking setting in the request, so thinking was left at the model's default; about 3 in 10 responses included thinking.

1127 * **What was measured:** The 20-issue triage job from [Trim input and context tokens](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#trim-input-and-context-tokens) and its long variant, run on Claude Sonnet 5 on August 19, 2026 (dated in UTC), three runs per setting, with no thinking setting in the request, so thinking was left at the model’s default; about 3 in 10 responses included thinking.

11281128 * **The chart:** The chart in [Manage the context lifecycle](https://platform.claude.com/docs/en/about-claude/models/optimizing-for-cost-and-intelligence#manage-the-context-lifecycle) compares four settings on each job: the job with prompt caching and input trimming (image resizing and tool search) on, and the same with one lever added, which is context editing, compaction, or the prune. The chart counts a change of under 10% as no change.

11291129 * **Lever settings:** The lever settings differed between the two jobs: context editing cleared at least 2,000 tokens once the prompt passed 20,000 on the 20-issue job, and at least 40,000 once it passed 50,000 on the long one, keeping the three most recent tool uses, and compaction triggered at 50,000 input tokens on the 20-issue job and 80,000 on the long one.

1130 * **How priced:** Each cost is the mean of the three runs, from the responses' usage fields at Claude Sonnet 5's list prices, with cache writes at $2.50 USD and cache reads at $0.20 USD per million tokens.

1130 * **How priced:** Each cost is the mean of the three runs, from the responses’ usage fields at Claude Sonnet 5’s list prices, with cache writes at $2.50 USD and cache reads at $0.20 USD per million tokens.

11311131 * **Results, input trimming:** A fifth setting, prompt caching with no input trimming, cost $0.57 USD a run on the 20-issue job and $1.38 USD on the long one, so input trimming took 26% off the first and 21% off the second. Against the same 20-issue job with caching off, which cost $3.36 USD a run (reference 16), prompt caching and input trimming together saved 88% (the unrounded means give 87.5%).

11321132 * **Results, 20-issue run:** On the 20-issue run, where the job with no lever cost $0.42 USD a run, context editing cost 73% more, compaction ran once near the end of each run and cost 7% less, and the prune rewrote nothing, because no search result was over its 2,000-character threshold.

11331133 * **Results, long run:** On the long run, where the job with no lever cost $1.09 USD a run, the prune cost 39% less, compaction 32% less, and context editing 4% more.

This page’s history 28 edits

27 Oct 2026 +186 −186 · this one
26 Oct 2026 +2 −2
23 Oct 2026 +49 −39
23 Oct 2026 +19 −20
22 Oct 2026 +5 −4
21 Oct 2026 +15 −15

The page’s own history →

Where it came from

The captureapi
Upstream pageplatform.claude.com

Around it

Every changethe ledger
That day27 Oct 2026