Follow Discord
Sweep 22 Oct 2026 · 04:32Z Build v2.1.308 527 read Stable v2.1.308 Latest v2.1.308 Next v2.1.308 Feeds RSS JSON llms.txt llms-full.txt Unofficial
HomeReleasesv2.1.224entry #55

Claude Code v2.1.224 ·

claude plugin eval writes a report and publishes it by default

Plugin eval now writes report.html and publishes it to claude.ai unless you pass --no-publish.

Use it now Notable New Features
JSON All of v2.1.224
Use it nowTier: how much it should matter to you
4Useful: my rating, 1 to 5
3Signal: worth watching, 1 to 5
Plugin EvalArea: what it touches
New FeaturesKind: in v2.1.224,
What probably matters to youSection of the release
What

After cases run, report.html is written into the results directory even without --report, and the report is published to claude.ai by default rather than only with --publish-report. Pass --no-publish to keep it local. --no-publish and --publish-report together are an error. Publish-by-default is controlled by a server flag whose built-in fallback is on, so with no server value it publishes; an explicit --publish-report ignores the flag entirely.

Details
  • The JSON result is now computed once and shared between writing, publishing and printing.
  • Publishing failures explain that artifacts are turned off for the account, provider or privacy mode, and the failure text names the local copy when one was written.
  • --publish-report's help text changed to match the new default.
Usage
claude plugin eval ./my-plugin --no-publish
Evidence

tengu_quartz_thimble, --no-publish and --publish-report are contradictory — pass at most one.

Strings lifted out of the shipped bundle, so the claim above can be checked against them.

Read from
Names in the bundleclaude plugin eval--no-publish--publish-report--report
Feature flag
tengu_quartz_thimble On for this account, and not off by default

The flag server returned on for the one account this site reads, and nothing in this release compiles it off by default. The compiled default is shown below, and says which it is when we cannot read one: a fifth of gates compile in a string or a number rather than on or off, and most published releases have no gate table behind them at all. No client can see what the server returns for your account.

This account: on · anonymous baseline: on · compiled default in v2.1.224: on

These values were read against a different version of Claude Code, so treat them as the nearest reading available instead of one taken on this release.

Read once, for one account on one subscription tier, against v2.1.224. It isn't a statement about your account. What a flag value here can and cannot tell you

What has happened since
Flag reading moved The flag server now returns on for tengu_quartz_thimble, read as this account. A reading is one sample. Claude Code evaluates its flags remotely, so no client sees the targeting rule behind a value and this says nothing about your account.
Flag reading moved The flag server now returns nothing at all, having stopped carrying it for tengu_quartz_thimble, read as this account. A reading is one sample. Claude Code evaluates its flags remotely, so no client sees the targeting rule behind a value and this says nothing about your account.

See this across every release →

How sure we are
One source agreesOne thing we can check says the same as this entry.
The name it cites is new in this buildNew in this build: --no-publish

See this entry in the whole of v2.1.224 →

Feedback