What probably matters to youSection of the release
The plugin eval command accepts a new --eval-dir <dir> flag, and the eval directory can also come from a plugin manifest's experimental.evals key; evals/ remains the fallback, and mis-shaped manifest values produce warnings that name the fix. Case discovery now screens the target path for trust before walking it and resolves it relative to the working directory. The error text shown when ablation runs with no plugin was rewritten to suggest targeting the plugin directory itself.
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Confirmed sinceAnthropic's documentation has since written up --eval-dir, on Plugins reference.| `experimental.evals` | string\|array | Directory below the plugin root that holds the plugin's [eval cases](/docs/en/plugin-evals#use-a-different-eval-directory), when it isn't the default `evals/`. `claude plugin eval --eval-dir` overri…plugins-referencesee the edit
Added sinceA small documentation edit on Create and distribute a plugin marketplace touched a line naming plugin eval after this was published.Test your marketplace before sharing. Validation checks file structure; to test whether a plugin changes what Claude does on realistic prompts, run its eval suite with [`claude plugin eval`](/docs/en/plugin-evals) before you publish a new …plugin-marketplacessee the edit
Confirmed sinceAnthropic's documentation has since written up plugin eval, on Claude Code changelog.* Added `claude plugin eval`: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see `claude plugin eval --help`changelogsee the edit
How sure we are
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agreesAnthropic's documentation has since written up plugin eval, on Claude Code changelog.
The name it cites is new in this buildNew in this build: --eval-dir