Follow Discord
Sweep 22 Oct 2026 · 04:32Z Build v2.1.308 527 read Stable v2.1.308 Latest v2.1.308 Next v2.1.308 Feeds RSS JSON llms.txt llms-full.txt Unofficial

Claude Code v2.1.269 ·

claude plugin eval reaches general availability with a trust-confirmation requirement

claude plugin eval is now generally available everywhere but requires trusting a plugin directory before it runs, with clearer help text and init delegation

Group of 5 Use it now Notable Improvements
JSON All of v2.1.269
Use it nowTier: how much it should matter to you
4Useful: my rating, 1 to 5
3Signal: worth watching, 1 to 5
Plugin EvalArea: what it touches
ImprovementsKind: in v2.1.269,
What probably matters to youSection of the release

What

  • claude plugin eval now reports itself as generally available for every client type (Bedrock/Vertex/Foundry, gateways, telemetry-disabled clients, CI), unless a server-side flag turns it off — in which case the CLI now says the command exists but is switched off, rather than claiming it doesn't exist.
  • Before loading a target, claude plugin eval now runs a trust check that must resolve to trusted: true — via an already-installed plugin, an existing trust marker, the --trust-plugin flag, or an interactive prompt — or the command errors out and exits without loading or running anything.
  • The command's help text was expanded to explain that it loads and runs a plugin's eval suite on your machine, that sandboxing limits but doesn't guarantee safety, and that the first run against an untrusted plugin directory will prompt for confirmation (use --trust-plugin to pre-answer that, e.g. for CI).
  • claude plugin eval init, when run without a terminal but detected as launched from inside another Claude Code session, can now delegate its authoring interview to that parent session instead of failing with "no TTY available". In that delegated interview flow, the interviewer must now explicitly ask whether you trust the plugin directory before piloting eval cases, and only adds --trust-plugin on an explicit yes; case files are still written on a no/no-answer, just not piloted.

Why These changes make claude plugin eval a supported, documented feature everywhere while making sure a plugin's eval code — which runs on your machine — isn't executed against an untrusted directory without your explicit confirmation.

Read from
Names in the bundleclaude plugin eval init
What the documentation says
claude plugin eval init Error reference modified, high confidence You ran [`claude plugin eval`](/docs/en/plugin-evals) or `claude plugin eval init` and it exited 1 with one of these messages before doing anything: see the edit
claude plugin eval init Plugins reference modified, high confidence claude plugin eval init [name] [options] see the edit
claude plugin eval init Test plugins with evals modified, high confidence You don't have to write the suite manually. `claude plugin eval init` asks you about your plugin, proposes the cases and graders, tries them, and writes the files. You can also ask Claude to do the same from a session you already have open. see the edit
Since it was published

The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.

Added since A small documentation edit on Test plugins with evals touched a line naming claude plugin eval init after this was published. * The same authentication and model provider your normal Claude Code sessions use. Eval runs, judge-scored graders, and `claude plugin eval init` call the model with your credentials, so they count against your plan's usage limits or your … plugin-evals see the edit
Confirmed since Anthropic's documentation has since written up claude plugin eval init, on What's new. **`claude plugin eval`**: run your plugin against a suite of test cases, score the results, and compare against a no-plugin baseline. `claude plugin eval init` drafts the cases and graders for you. whats-new/index see the edit
How sure we are
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agreesclaude plugin eval init on Error reference
Anthropic's release notes agreeAdded claude plugin eval: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see…

See this entry in the whole of v2.1.269 →

Feedback