Source Intelligence
Follow Discord

DisclaimerUnofficial, and not affiliated with Anthropic. Nearly all of this is read straight out of what ships: npm bundles, captured prompts, published docs. Anthropic's own notes go in verbatim, marked as theirs. The rest is my reading, and every entry carries the strings behind it. If one looks wrong, vote it down and say why.

← All of v2.1.242 Home All releases ←olderv2.1.241 v2.1.243newer→
Claude Code v2.1.242

plugin eval can mock a plugin's MCP servers, with docs to match

Use it now
Useful4 Signal4
Plugin Eval Notable not in their notes

Plugin evals can swap a plugin's MCP servers for fakes defined in a mocks folder.

--mocksplugin eval
What

Evaluation runs of a plugin can now swap that plugin's real MCP servers, the external tool servers a plugin declares, for fake ones defined as files under <eval dir>/mocks/, so an eval exercises tool calls without starting anything real. The mode is controlled by a new --mocks flag that defaults to record, and the bundled quick reference for plugin eval now carries a Mocks section describing the whole layout. The commands remain early access.

Details
  • --mocks record (the default) serves canned responses from mocks/<server>/<tool>.md; --mocks off spawns the real servers instead; any other value exits with "Error: --mocks must be record or off".
  • A responder file may carry an expect: guard on the input, which aborts the run with a record of the server, tool and reason when the call does not match.
  • Responders also support error: true to return a failure, and type: agent with its own abort conditions.
  • mocks/<server>/_tools.json supplies the saved tool list; without it a tool is served with a permissive schema and no description.
  • _server.md lets a single responder answer several tools for that server.
  • Directory and tool names must be valid tool-name segments, __ is rejected because it separates server name from tool name, and a directory is capped at 256 entries.
  • Mocks can shadow a server that the plugins under test declare, or stand alone with no real counterpart.
  • Grading gains mock_calls as a source, so a grader can assert on which mock tools were called, and the LLM judge can now grade image files.
  • The reference synopsis also lists --report, --publish-report and --no-publish.
  • Enabled per organization: when your organization is not enabled, the commands print an early-access line and exit with status 1.
Evidence

Mock stand-ins for MCP servers, from <eval dir>/mocks/ (record | off; default: record, --mocks record|off

Strings lifted out of the shipped bundle, so the claim above can be checked against them.

Related

Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.

See this entry in the whole of v2.1.242 →