What
claude plugin evalnow reports itself as generally available for every client type (Bedrock/Vertex/Foundry, gateways, telemetry-disabled clients, CI), unless a server-side flag turns it off — in which case the CLI now says the command exists but is switched off, rather than claiming it doesn't exist.- Before loading a target,
claude plugin evalnow runs a trust check that must resolve totrusted: true— via an already-installed plugin, an existing trust marker, the--trust-pluginflag, or an interactive prompt — or the command errors out and exits without loading or running anything. - The command's help text was expanded to explain that it loads and runs a plugin's eval suite on your machine, that sandboxing limits but doesn't guarantee safety, and that the first run against an untrusted plugin directory will prompt for confirmation (use
--trust-pluginto pre-answer that, e.g. for CI). claude plugin eval init, when run without a terminal but detected as launched from inside another Claude Code session, can now delegate its authoring interview to that parent session instead of failing with "no TTY available". In that delegated interview flow, the interviewer must now explicitly ask whether you trust the plugin directory before piloting eval cases, and only adds--trust-pluginon an explicit yes; case files are still written on a no/no-answer, just not piloted.
Why
These changes make claude plugin eval a supported, documented feature everywhere while making sure a plugin's eval code — which runs on your machine — isn't executed against an untrusted directory without your explicit confirmation.
Names in the bundleclaude plugin eval init
Documented inclaude-code/whats-new/index
claude plugin eval init
Error reference modified, high confidence
You ran [`claude plugin eval`](/docs/en/plugin-evals) or `claude plugin eval init` and it exited 1 with one of these messages before doing anything:see the edit
claude plugin eval init
Plugins reference modified, high confidence
claude plugin eval init [name] [options]see the edit
claude plugin eval init
Test plugins with evals modified, high confidence
You don't have to write the suite manually. `claude plugin eval init` asks you about your plugin, proposes the cases and graders, tries them, and writes the files. You can also ask Claude to do the same from a session you already have open.see the edit
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Added since
A small documentation edit on Test plugins with evals touched a line naming claude plugin eval init after this was published.
* The same authentication and model provider your normal Claude Code sessions use. Eval runs, judge-scored graders, and `claude plugin eval init` call the model with your credentials, so they count against your plan's usage limits or your …plugin-evals see the edit
Confirmed since
Anthropic's documentation has since written up claude plugin eval init, on What's new.
**`claude plugin eval`**: run your plugin against a suite of test cases, score the results, and compare against a no-plugin baseline. `claude plugin eval init` drafts the cases and graders for you.whats-new/index see the edit
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agrees
claude plugin eval init on Error reference
Anthropic's release notes agree
Added claude plugin eval: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see…