You'll noticeTier: how much it should matter to you
1Useful: my rating, 1 to 5
1Signal: worth watching, 1 to 5
Plugin EvalArea: what it touches
ImprovementsKind: in v2.1.269,
ImprovementsSection of the release
What
The --max-cost-usd option in the eval/grading CLI (used to grade plugin evaluation runs) has updated help text. It now explains that the cost ceiling is checked before each run launches, so any spending overrun is bounded to whatever runs are currently in flight — either one run, or up to the number set by --concurrency. The previous wording said overrun was bounded to a single agent run.
Why
This clarifies that when running evaluations concurrently, the actual maximum spend can exceed the --max-cost-usd ceiling by more than one run's worth, which matters for anyone relying on this flag as a hard budget cap.
--max-cost-usdPlugins referencemodified, high confidence| `--max-cost-usd <usd>` | Stop before the next run once spend reaches this, exit 2, and report partial results | No ceiling |see the edit
Since it was published
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Confirmed sinceAnthropic's documentation has since written up --max-cost-usd, on Test plugins with evals.| `--max-cost-usd <usd>` | No ceiling | A ceiling on the run's list-price cost estimate, not on plan usage. Checked before each run starts. Once spent, nothing further starts; runs that already started finish, so spend can pass the ceiling…plugin-evalssee the edit
How sure we are
One source agreesOne thing we can check says the same as this entry.
Anthropic's documentation agrees--max-cost-usd on Plugins reference
