Plugin eval now writes report.html and publishes it to claude.ai unless you pass --no-publish.
tengu_quartz_thimble Off by default, switched on for this accountThe shipped code defaults this off, and the flag server returned on for the one account this site reads on this version. That is the reading that makes the entry above worth a second look, and it still says nothing about your account.
This account: on · anonymous baseline: on · compiled default in v2.1.224: off
These values were read against a different version of Claude Code, so treat them as the nearest reading available instead of one taken on this release.
Read once, for one account on one subscription tier, against v2.1.224. It isn't a statement about your account. What a flag value here can and cannot tell you
What's wrong with this entry?
After cases run, report.html is written into the results directory even without --report, and the report is published to claude.ai by default rather than only with --publish-report. Pass --no-publish to keep it local. --no-publish and --publish-report together are an error. Publish-by-default is controlled by a server flag whose built-in fallback is on, so with no server value it publishes; an explicit --publish-report ignores the flag entirely.
- The JSON result is now computed once and shared between writing, publishing and printing.
- Publishing failures explain that artifacts are turned off for the account, provider or privacy mode, and the failure text names the local copy when one was written.
--publish-report's help text changed to match the new default.
claude plugin eval ./my-plugin --no-publishtengu_quartz_thimble, --no-publish and --publish-report are contradictory — pass at most one.
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
Related
Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.
-
v2.1.233
Eval report publishing prints a working link and a way to opt out
Both mention eval publish