Source Intelligence

DisclaimerUnofficial, and not affiliated with Anthropic. Nearly all of this is read straight out of what ships: npm bundles, captured prompts, published docs. Anthropic's own notes go in verbatim, marked as theirs. The rest is my reading, and every entry carries the strings behind it. If one looks wrong, vote it down and say why.

All of v2.1.251 Home All releases olderv2.1.250
Claude Code v2.1.251

Eval-authoring guidance now requires mocking MCP servers

You'll notice
Useful2 Signal2
Skills

The eval-writing skill now requires stubbing MCP calls so evals never hit real services.

What

The built-in skill for writing evals gained a required step telling authors to stub every MCP tool call an eval can make, so evals never reach real MCP services.

Details
  • each tool gets a mock file under mocks/<server>/<tool>.md
  • flows that make several calls can use a single _server.md agent mock with an abort_when: list
Evidence

Step 3a — Mock the MCP servers

Strings lifted out of the shipped bundle, so the claim above can be checked against them.

See this entry in the whole of v2.1.251 →