--allow-real-servers: when combined with --mocks record, lets a plugin's real MCP server processes start and run for any servers that have no mock available, running outside Claude Code's OS sandbox (a restricted execution environment)
The --mocks help text was also expanded to better explain the difference between record and off behavior. Separately, when a plugin's MCP servers can't be enumerated (for example, an invalid manifest, archive, or MCPB bundle), setting allowRealServers now runs the real servers instead of refusing or mocking them; without it, an invalid manifest now causes affected cases to be refused outright in record mode, rather than just noted.
Why
These flags make it practical to run plugin evals unattended in automated pipelines and to test against real MCP servers when no mock exists, at the cost of running outside the sandbox's protections.
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Added sinceA small documentation edit on Test plugins with evals touched a line naming --trust-plugin after this was published.The first time you run `claude plugin eval` against a directory, Claude Code asks `Trust this plugin directory?` before it loads anything from it, unless you already accepted the trust prompt there in an interactive `claude` session. Insid…plugin-evalssee the edit
Confirmed sinceAnthropic's documentation has since written up --trust-plugin, on Test plugins with evals.| `--trust-plugin` | Off | Skip the first-run trust prompt for a plugin whose code and suite you'd run yourself. Pass it in CI so the job is never refused by or left waiting at the prompt. See [What a run can access](#security) |plugin-evalssee the edit
Confirmed sinceAnthropic's documentation has since written up --allow-real-servers, on Test plugins with evals.When a case asked for a tool you didn't grant, the progress output lists it as `not granted`. Tools on a [mocked](#mock-mcp-servers) MCP server need no grant. Tools on a real plugin MCP server need both the server started, with `--allow-re…plugin-evalssee the edit
How sure we are
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agreesAnthropic's documentation has since written up --allow-real-servers, on Test plugins with evals.
Anthropic's release notes agreeAdded claude plugin eval: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see…
The name it cites is new in this buildNew in this build: --allow-real-servers
