{"name":"plugin eval","slug":"plugin-eval","family":"cli","title":"CLI flags","noun":"CLI flag","description":"Plugin eval now writes report.html and publishes it to claude.ai unless you pass --no-publish.","description_source":"entry","described_by":{"version":"2.1.224","anchor":"claude-plugin-eval-writes-a-report-and-publishes-it-by-defau"},"presence":{"first_seen":null,"removed_in":null,"in_current_build":false,"newest_mined":"2.1.308","builds":0,"mined_builds":149,"first_cited":null},"aliases":[],"entries":[{"version":"2.1.305","anchor":"plugin-eval-can-pass-the-session-api-key-to-its-test-runs-by","heading":"claude plugin eval passes your session's API key to its test runs","line":"With an API key session, `claude plugin eval` now hands the key to its test runs itself, except on Windows, where you must set `ANTHROPIC_API_KEY`","released_at":"2026-10-20 19:29:36","reason":"named in this entry"},{"version":"2.1.289","anchor":"claude-plugin-eval-resolves-installed-plugins-from-marketpla","heading":"claude plugin eval tests folder marketplace plugins from where sessions load them","line":"Evaluating an installed plugin from a folder marketplace now runs it from that folder, or stops with an error if the folder no longer lists it","released_at":"2026-10-03 20:12:02","reason":"named in this entry"},{"version":"2.1.283","anchor":"claude-plugin-eval-requires-git-231-or-newer","heading":"claude plugin eval now needs git 2.31 or newer","line":"`claude plugin eval` now checks your git version and refuses to run with git older than 2.31","released_at":"2026-09-25 18:46:11","reason":"named in this entry"},{"version":"2.1.269","anchor":"claude-plugin-eval-ga-text-and-its-server-side-kill-switch","heading":"claude plugin eval GA text and its server-side kill switch","line":"claude plugin eval is now generally available, with a server-side switch that can turn it off per session","released_at":"2026-09-11 18:12:49","reason":"named in this entry"},{"version":"2.1.269","anchor":"claude-plugin-eval-now-requires-explicit-trust-confirmatio","heading":"claude plugin eval now requires explicit trust confirmation before running","line":"`claude plugin eval` now checks that a plugin is trusted before it will run its eval suite","released_at":"2026-09-11 18:12:49","reason":"named in this entry"},{"version":"2.1.269","anchor":"plugin-eval-cli-help-text-expanded-with-sandboxingtrust-dis","heading":"plugin eval CLI help text expanded with sandboxing\/trust disclosure","line":"`claude plugin eval` help text now explains sandboxing limits and the new untrusted-plugin confirmation prompt","released_at":"2026-09-11 18:12:49","reason":"named in this entry"},{"version":"2.1.251","anchor":"eval-runs-clean-up-the-whole-process-tree","heading":"Eval runs clean up the whole process tree","line":"Plugin eval kills the whole process tree so spawned tools don't outlive the run.","released_at":"2026-08-28 15:34:26","reason":"named in this entry"},{"version":"2.1.242","anchor":"plugin-eval-can-mock-a-plugin-s-mcp-servers-with-docs-to-mat","heading":"plugin eval can mock a plugin's MCP servers, with docs to match","line":"Plugin evals can swap a plugin's MCP servers for fakes defined in a mocks folder.","released_at":"2026-08-24 19:16:14","reason":"named in this entry"},{"version":"2.1.234","anchor":"plugin-eval-no-longer-swallows-piped-input","heading":"plugin eval no longer swallows piped input","line":"Piping input into `plugin eval` now works because startup no longer captures that stdin first.","released_at":"2026-08-17 18:19:13","reason":"named in this entry"},{"version":"2.1.234","anchor":"plugin-eval-reference-documents-image-grading-and-new-exit-c","heading":"Plugin eval reference documents image grading and new exit codes","line":"Plugin eval docs now say image focus files are shown to the judge, and document new exit codes.","released_at":"2026-08-17 18:19:13","reason":"named in this entry"}],"entries_total":22,"docs":[{"source":"claude-code","path":"changelog","title":"Claude Code changelog","documented":false},{"source":"claude-code","path":"errors","title":"Error reference","documented":false},{"source":"claude-code","path":"plugin-evals","title":"Test plugins with evals","documented":false},{"source":"claude-code","path":"plugins\/cli-reference","title":"Plugin commands reference","documented":false},{"source":"claude-code","path":"plugins\/create","title":"Create a Claude Code plugin","documented":false},{"source":"claude-code","path":"plugins\/manifest-reference","title":"Plugin manifest reference","documented":false},{"source":"claude-code","path":"plugins\/publish","title":"Publish and distribute a plugin","documented":false},{"source":"claude-code","path":"plugins\/troubleshooting","title":"Troubleshoot plugins","documented":false},{"source":"claude-code","path":"skills","title":"Extend Claude with skills","documented":false},{"source":"claude-code","path":"sub-agents","title":"Create custom subagents","documented":false}],"docs_total":12,"gates":[],"related":[{"name":"plugin","slug":"plugin","description":"Telemetry values now pass through marker helpers that declare them safe; nothing sent actually changed."},{"name":"plugin configure","slug":"plugin-configure","description":"New `claude plugin configure <plugin>` shows a plugin's options, or saves values piped in as JSON with `--values-stdin`"},{"name":"plugin details","slug":"plugin-details","description":"Plugin subcommand descriptions come from the shared table that also hides early-access entries."},{"name":"plugin disable","slug":"plugin-disable","description":"Plugin commands now explain refusals for built-ins and org-locked plugins instead of acting oddly."},{"name":"plugin disable-all","slug":"plugin-disable-all","description":"Plugin disable, disable-all, and update commands gain --json output alongside other plugin commands"},{"name":"plugin enable","slug":"plugin-enable","description":"Plugin subcommand descriptions come from the shared table that also hides early-access entries."},{"name":"plugin eval init","slug":"plugin-eval-init","description":"Eval scaffolding refuses to create files outside a plugin or skill folder unless you pass --eval-dir."},{"name":"plugin init","slug":"plugin-init","description":"The check at the end of `claude plugin init` now warns, rather than failing, when a name cannot be recorded"},{"name":"plugin install","slug":"plugin-install","description":"Plugin install and update take -y to skip the command confirmation, but not inside a session."},{"name":"plugin install <plugin>@<name>","slug":"plugin-install-plugin-name","description":null}],"url":"https:\/\/changelogs.core-directive.com\/reference\/cli\/plugin-eval","family_url":"https:\/\/changelogs.core-directive.com\/reference\/cli.json"}