Source Intelligence

DisclaimerUnofficial, and not affiliated with Anthropic. Nearly all of this is read straight out of what ships: npm bundles, captured prompts, published docs. Anthropic's own notes go in verbatim, marked as theirs. The rest is my reading, and every entry carries the strings behind it. If one looks wrong, vote it down and say why.

All of v2.1.224 Home All releases olderv2.1.223 v2.1.225newer

Eval-authoring guidance had two wrong field names

Use it now
Useful2 Signal2
Plugins not in their notes

Plugin eval guidance now names the real fields, suite.plugins and costUsd.

suite.pluginscostUsdaggregate-result.json
What

The guidance for writing plugin evals now names the fields that actually exist: suite.plugins for checking the plugin loaded, and costUsd for the cost figure in aggregate-result.json. It previously said plugins and cost_usd.

Evidence

The pilot's top-level \costUsd\ in \aggregate-result.json\``

Strings lifted out of the shipped bundle, so the claim above can be checked against them.

See this entry in the whole of v2.1.224 →