{"version":"2.1.242","anchor":"eval-results-record-which-mocks-answered-and-why-a-run-abort","canonical_anchor":"eval-results-record-which-mocks-answered-and-why-a-run-abort","heading":"Eval results record which mocks answered and why a run aborted","tier":"internal","area":"Plugin Eval","scope":null,"heads_up":null,"url":"https:\/\/changelogs.core-directive.com\/v\/2.1.242\/e\/eval-results-record-which-mocks-answered-and-why-a-run-abort","release_url":"https:\/\/changelogs.core-directive.com\/v\/2.1.242","markdown":"### Eval results record which mocks answered and why a run aborted\n\nEval results now list which stand-in servers answered and count calls nothing covered.\n\n**What**\n\nAn eval result now includes a mocks block listing each stand-in server's kind and per-tool responder, defaulting to \"fixed\", plus a tally that counts calls the mocks did not cover as `unmocked`.\n\n**Details**\n\n- A run that aborted, or whose mocks could not be prepared, returns a zero score carrying the server, tool and reason, plus any `auth_rejected` flag.\n- Mocked tools are announced per case and excluded from the check that a case exercises a real tool.\n- Mock call logs are read back into the result as call records and a tally.\n- A separate standalone eval config schema, covering schema version, graders and config overrides, was dropped from this part of the bundle.\n\n**Evidence**\n\n`mocks: could not be prepared for this case`, `mockSetupFailure`, `\"Mock stand-ins for MCP servers, from <eval dir>\/mocks\/ (record | off; default: record \u2014 off spawns the real servers, gated by --allow-tools as usual)\"`\n\n- Area: Plugin Eval\n- Tier: Under the hood\n- Useful: 1\/5\n- Signal: 2\/5"}