You'll noticeTier: how much it should matter to you
2Useful: my rating, 1 to 5
2Signal: worth watching, 1 to 5
Plugin EvalArea: what it touches
ImprovementsKind: in v2.1.269,
What probably matters to youSection of the release
What
The claude plugin eval command normally publishes its report automatically. Now, if the run looks like it was started by another Claude Code session rather than a person directly (detected via the CLAUDE_CODE_CHILD_SESSION environment variable, or CLAUDECODE combined with no interactive terminal), the report is kept local by default instead. The CLI prints a note that --publish-report must be passed explicitly to publish it in that case.
Why
This stops eval reports from being auto-published when Claude Code itself is the one running the eval (for example, as part of an automated or nested workflow), keeping publication reserved for runs a person actually initiated.
tengu_quartz_thimbleOn for this account, and not off by default
The flag server returned on for the one account this site reads, and nothing in this release compiles it off by default. The compiled default is shown below, and says which it is when we cannot read one: a fifth of gates compile in a string or a number rather than on or off, and most published releases have no gate table behind them at all. No client can see what the server returns for your account.
This account: on · anonymous baseline: on · compiled default in v2.1.269: on
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Confirmed sinceAnthropic's documentation has since written up --publish-report, on Test plugins with evals.| `--publish-report` | | Publish the report even where it would stay local by default, such as a run a Claude Code session started |plugin-evalssee the edit
How sure we are
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agreesAnthropic's documentation has since written up --publish-report, on Test plugins with evals.
Anthropic's release notes agreeAdded claude plugin eval: run a plugin's eval suite against Claude Code and get scored, reproducible results (JSON + HTML report); see…
