Source Intelligence

DisclaimerUnofficial, and not affiliated with Anthropic. Nearly all of this is read straight out of what ships: npm bundles, captured prompts, published docs. Anthropic's own notes go in verbatim, marked as theirs. The rest is my reading, and every entry carries the strings behind it. If one looks wrong, vote it down and say why.

All of v2.1.224 Home All releases olderv2.1.223 v2.1.225newer

plugin eval: report publishing flags reworked

Use it now
Useful4 Signal2
Plugin Eval Notable not in their notes

Plugin eval publishes its HTML report by default; --publish-report now means require publishing.

--publish-report--no-publish--reportclaude plugin eval
What

claude plugin eval now publishes the HTML report by default when your account supports it. --publish-report changed meaning to "require publishing", and explains why if it is unavailable; --no-publish keeps the report local. Passing both is now an error. --report now means the report replaces the results directory, implying a default location.

Details
  • Results from the LLM judge on long inputs now append an advisory suggesting a regex grader instead.
Evidence

Error: --no-publish and --publish-report are contradictory — pass at most one.

Strings lifted out of the shipped bundle, so the claim above can be checked against them.

Related

Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.

See this entry in the whole of v2.1.224 →