`plugin eval`

A Claude Code CLI flag, read out of the shipped bundle. No mined inventory carries it: everything below comes from what this site has written about it.

What it is

Plugin eval now writes report.html and publishes it to claude.ai unless you pass --no-publish.

Not Anthropic's. This is the line from the earliest changelog entry here that named it, v2.1.224.

Presence across releases

The miner has never read a build containing this name. It is here because a changelog entry of ours names it, which is evidence that it existed and not evidence of which releases carried it. The inventory starts at v2.1.138 and everything older than that was never mined.

Changelog entries naming it

v2.1.242plugin eval can mock a plugin's MCP servers, with docs to match
Plugin evals can swap a plugin's MCP servers for fakes defined in a mocks folder.

named in this entry
v2.1.235plugin eval can be pointed at a different case directory
Point plugin eval at any case directory with --eval-dir, still behind an early-access gate or env var.

named in this entry
v2.1.235plugin eval now tells you when a nearby plugin is not being loaded
Plugin eval now tells you when it deliberately skipped loading a nearby plugin it doesn't trust.

named in this entry
v2.1.235--eval-dir on the eval command, with warnings that name the fix
The eval command reads its directory from a flag or the manifest, warning clearly when the value is malformed.

named in this entry
v2.1.235Eval runs record their output directory and show per-case advisories
Eval runs record which directory they used and show per-case warning notes in the HTML report.

named in this entry
v2.1.234plugin eval no longer swallows piped input
Piping input into plugin eval now works because startup no longer captures that stdin first.

named in this entry
v2.1.234Plugin eval reference documents image grading and new exit codes
Plugin eval docs now say image focus files are shown to the judge, and document new exit codes.

named in this entry
v2.1.234claude plugin eval handles termination signals and cleans up its output
Plugin eval runs now handle kill signals with proper exit codes and title reports with plugin names.

named in this entry
v2.1.234Plugin eval reference rewritten around a tighter sandbox
Plugin eval runs now happen in a tighter sandbox with their own git repo and home directory.

named in this entry
v2.1.233Plugin eval runs report exit codes, stderr and auth rejection
Plugin eval runs now report exit codes and stderr tails when a case fails.

named in this entry
v2.1.233Plugin eval checks your credentials before running any case
Plugin eval checks your credentials first and aborts instead of failing every case.

named in this entry
v2.1.233Plugin eval stops the suite when authentication fails
A failed login stops the whole eval suite instead of wasting every case.

named in this entry
v2.1.233plugin eval --verbose now writes traces to the debug log
Eval --verbose now writes traces to the debug log instead of streaming them live.

named in this entry
v2.1.233Eval report publishing prints a working link and a way to opt out
Published eval reports now print a working link and mention --no-publish.

named in this entry
v2.1.233Plugin eval tells the model the env var that turns it on
Setting CLAUDE_CODE_WALNUT_SPIRE=1 turns on the early-access plugin eval command.

named in this entry
v2.1.233The built-in guide agent can answer questions about plugin eval and /skill-doctor offline
The built-in guide agent can now answer about plugin eval and /skill-doctor from bundled reference text.

named in this entry
v2.1.224claude plugin eval writes a report and publishes it by default
Plugin eval now writes report.html and publishes it to claude.ai unless you pass --no-publish.

named in this entry
v2.1.224plugin eval: report publishing flags reworked
Plugin eval publishes its HTML report by default; --publish-report now means require publishing.

named in this entry
v2.1.224Plugin eval can validate and report its own JSON output
Plugin eval JSON output is validated before printing, and suppressed with a stderr warning if invalid.

named in this entry

Documentation pages

None of Anthropic's documentation pages this site has a copy of carries this name.

Names beside it

Every CLI flag in the inventory starting plugin, up to twelve of them.

pluginFailed plugin commands now flush their analytics before exiting instead of dropping the event. never mined plugin disablePlugin commands now explain refusals for built-ins and org-locked plugins instead of acting oddly. never mined plugin enablePlugin subcommand descriptions come from the shared table that also hides early-access entries. never mined plugin eval initPoint plugin eval at any case directory with --eval-dir, still behind an early-access gate or env var. never mined plugin installPlugin install and update take -y to skip the command confirmation, but not inside a session. never mined plugin listclaude plugin list now groups plugins by scope, including a "Synced from claude.ai" section. never mined plugin uninstallPlugin commands now explain refusals for built-ins and org-locked plugins instead of acting oddly. never mined plugin updatePlugins that build via a command won't silently run it; you get an error telling you to update. never mined plugin validateclaude plugin validate now falls back to checking skills, agents and commands when there is no manifest. never mined

Read out of the published npm bundle release by release, and out of Anthropic's own documentation as this site captured it. Nothing on this page is a description anybody here wrote about what the CLI flag does. All CLI flags.