What
If your organisation's allowedMcpServers or deniedMcpServers settings would block the stand-in servers, the eval run stops with a policy-blocked error instead of quietly starting nothing.
Details
- The message advises allowlisting the stand-in command for eval runs, or passing
--mocks off.
Evidence
mocks: stand-in blocked by managed MCP policy
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Added since
A small documentation edit on Control MCP server access for your organization touched a line naming deniedMcpServers after this was published.
* [Server-managed settings](/docs/en/server-managed-settings): deliver `allowedMcpServers` and `deniedMcpServers` from the claude.ai admin consolemanaged-mcp see the edit
Added since
A small documentation edit on Control MCP server access for your organization touched a line naming allowedMcpServers after this was published.
* [Server-managed settings](/docs/en/server-managed-settings): deliver `allowedMcpServers` and `deniedMcpServers` from the claude.ai admin consolemanaged-mcp see the edit
Confirmed since
Anthropic's documentation has since written up deniedMcpServers, on Claude Code changelog.
* Fixed `allowManagedMcpServersOnly`, `deniedMcpServers` and `disableClaudeAiConnectors` set via MDM or `managed-settings.json` being ignored when server-managed settings are also presentchangelog see the edit
Confirmed since
Anthropic's documentation has since written up allowedMcpServers, on All settings.
| Restriction allowlists | Takes the list whole from the highest source that sets it, without adding entries from lower sources. When the highest source doesn't set one, takes it whole from the next source down | [`availableModels`](#avail…settings-reference see the edit
Confirmed since
Anthropic's documentation has since written up --mocks, on Plugins reference.
| `--mocks <mode>` | `record` or `off`. See [Mock MCP servers](/docs/en/plugin-evals#mock-mcp-servers) | `record` |plugins-reference see the edit
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agrees
Anthropic's documentation has since written up --mocks, on Plugins reference.
The name it cites is new in this build
New in this build: --mocks