DisclaimerUnofficial, and not affiliated with Anthropic. Nearly all of this is read straight out of what ships: npm bundles, captured prompts, published docs. Anthropic's own notes go in verbatim, marked as theirs. The rest is my reading, and every entry carries the strings behind it. If one looks wrong, vote it down and say why.
Eval-authoring guidance now requires mocking MCP servers
You'll notice
Useful2Signal2
Skills
The eval-writing skill now requires stubbing MCP calls so evals never hit real services.
What's wrong with this entry?
Anonymous. No account, no email.
What
The built-in skill for writing evals gained a required step telling authors to stub every MCP tool call an eval can make, so evals never reach real MCP services.
Details
each tool gets a mock file under mocks/<server>/<tool>.md
flows that make several calls can use a single _server.md agent mock with an abort_when: list
Evidence
Step 3a — Mock the MCP servers
Strings lifted out of the shipped bundle, so the claim above can be checked against them.