Auth-failure abort and report banner for plugin eval, behind the early-access check.
An eval run that cannot authenticate, or whose credential is rejected before any case completes, now aborts the whole suite instead of burning every remaining case on the same broken credential. The HTML report banner explains the stop, the CLI prints "stopped: authentication failed", and the outcome is reported separately from the two existing partial reasons, a cost ceiling and interruption. plugin eval remains behind an early-access check.
- Individual case results carry an auth-rejected outcome with its own label in the run-status table.
- The suite aborts once the first run comes back auth-rejected.
- A new environment variable marks the child process spawned for the
plugin eval initinterview, and its presence suppresses report publishing. - The command's long description moved into the shared command-help table.
A run could not authenticate (or the credential was rejected before any run), so the suite stopped before every case finished., a run could not authenticate \u2014 every remaining run would fail the same way, so the suite stops here. Fix the credential above and re-run., stopped: authentication failed
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Test your marketplace before sharing. Validation checks file structure; to test whether a plugin changes what Claude does on realistic prompts, run its eval suite with [`claude plugin eval`](/docs/en/plugin-evals) before you publish a new …plugin-marketplaces see the edit
* Added `claude plugin eval`: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see `claude plugin eval --help`changelog see the edit
Anthropic's documentation has since written up plugin eval, on Claude Code changelog.