Sweep 22 Sep 2026 · 15:52Z Build v2.1.280 501 read Stable v2.1.267 Latest v2.1.280 Next v2.1.280 Feeds RSS JSON llms.txt Unofficial
Writing the changelog v2.1.280 merge candidates · 1/5 waiting for the next tick
Claude Code v2.1.269 ·

claude plugin eval reaches general availability with a trust-confirmation requirement

claude plugin eval is now generally available everywhere but requires trusting a plugin directory before it runs, with clearer help text and init delegation

TierYou'll noticehow much it should matter to you
Useful3my rating, 1 to 5
Signal3worth watching, 1 to 5
AreaPlugin Evalwhat it touches
KindImprovementsin v2.1.269,
Group of 5 You'll notice

claude plugin eval is now generally available everywhere but requires trusting a plugin directory before it runs, with clearer help text and init delegation

What

  • claude plugin eval now reports itself as generally available for every client type (Bedrock/Vertex/Foundry, gateways, telemetry-disabled clients, CI), unless a server-side flag turns it off — in which case the CLI now says the command exists but is switched off, rather than claiming it doesn't exist.
  • Before loading a target, claude plugin eval now runs a trust check that must resolve to trusted: true — via an already-installed plugin, an existing trust marker, the --trust-plugin flag, or an interactive prompt — or the command errors out and exits without loading or running anything.
  • The command's help text was expanded to explain that it loads and runs a plugin's eval suite on your machine, that sandboxing limits but doesn't guarantee safety, and that the first run against an untrusted plugin directory will prompt for confirmation (use --trust-plugin to pre-answer that, e.g. for CI).
  • claude plugin eval init, when run without a terminal but detected as launched from inside another Claude Code session, can now delegate its authoring interview to that parent session instead of failing with "no TTY available". In that delegated interview flow, the interviewer must now explicitly ask whether you trust the plugin directory before piloting eval cases, and only adds --trust-plugin on an explicit yes; case files are still written on a no/no-answer, just not piloted.

Why These changes make claude plugin eval a supported, documented feature everywhere while making sure a plugin's eval code — which runs on your machine — isn't executed against an untrusted directory without your explicit confirmation.

Read from
Names in the bundleclaude plugin eval
Feature flag
tengu_sharded_snowflake Off in both readings

The flag server returned off for the account this site reads and for the anonymous baseline. A reading of off cannot rule out a rollout these two readings sit outside of.

This account: off · anonymous baseline: off · compiled default in v2.1.269: off

Read once, for one account on one subscription tier, against v2.1.269. It isn't a statement about your account. What a flag value here can and cannot tell you

What the documentation says
claude plugin eval Claude Code changelog modified, high confidence * Added `claude plugin eval`: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see `claude plugin eval --help` see the edit
claude plugin eval Error reference modified, high confidence You ran [`claude plugin eval`](/docs/en/plugin-evals) or `claude plugin eval init` and it exited 1 with one of these messages before doing anything: see the edit
claude plugin eval Plugins reference modified, high confidence | `experimental.evals` | string\|array | Directory below the plugin root that holds the plugin's [eval cases](/docs/en/plugin-evals#use-a-different-eval-directory), when it isn't the default `evals/`. `claude plugin eval --eval-dir` overri… see the edit
claude plugin eval Test plugins with evals modified, high confidence You don't have to write the suite manually. `claude plugin eval init` asks you about your plugin, proposes the cases and graders, tries them, and writes the files. You can also ask Claude to do the same from a session you already have open. see the edit
claude plugin eval Create and distribute a plugin marketplace modified, medium confidence Test your marketplace before sharing. Validation checks file structure; to test whether a plugin changes what Claude does on realistic prompts, run its eval suite with [`claude plugin eval`](/docs/en/plugin-evals) before you publish a new … see the edit
claude plugin eval Create plugins modified, medium confidence Trying the plugin with `--plugin-dir` tells you it can work. To find out how often Claude actually reaches for it and gets the right result, run it against a set of test prompts with [`claude plugin eval`](/docs/en/plugin-evals). Each prom… see the edit
claude plugin eval Extend Claude with skills modified, medium confidence Two tools automate that comparison. For a skill that ships in a [plugin](/docs/en/plugins), [`claude plugin eval`](/docs/en/plugin-evals) runs each prompt in an isolated session with and without the plugin, scores it with graders you defin… see the edit
Since it was published

The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.

Confirmed since Anthropic's documentation has since written up claude plugin eval, on What's new. **`claude plugin eval`**: run your plugin against a suite of test cases, score the results, and compare against a no-plugin baseline. `claude plugin eval init` drafts the cases and graders for you. whats-new/index see the edit
How sure we are
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agreesclaude plugin eval on Claude Code changelog
Anthropic's release notes agreeAdded claude plugin eval: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see…
The name it cites is new in this buildNew in this build: tengu_sharded_snowflake

See this entry in the whole of v2.1.269 →