Under the hood
Plugin eval mock runs can now flag an MCP server as 'withheld' in results
What
When running a mock evaluation of a plugin, the summary of results for each MCP server can now include a withheld: true flag, in addition to the existing replayPinned count.
Why
This lets plugin developers see when a server was withheld during a mock eval run, giving clearer insight into what happened during testing.
Something disagreesSomething we can check disagrees with this entry, or the writer said they could not settle it.
The writer flagged doubt
What causes a server to be marked withheld, and what practical effect it has on the eval, isn't stated.