Plugin eval runs report exit codes, stderr and auth rejection / Claude Code v2.1.233

Not switched on

Useful2 Signal2

Plugin Eval not in their notes

Plugin eval runs now report exit codes and stderr tails when a case fails.

Richer eval failure reporting sits behind the plugin eval early-access gate.

claude plugin eval

What

Eval run summaries now separate ok, error and exit outcomes. A non-zero exit produces an error built from the result message, or from the exit code plus the tail of stderr, falling back to "(no stderr)" when there is none. Behind the plugin eval early-access gate.

Details

Runs where a top-level assistant message reported authentication_failed or oauth_org_not_allowed are now flagged as auth-rejected rather than as generic errors.

Evidence

(no stderr)

Strings lifted out of the shipped bundle, so the claim above can be checked against them.

Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.

v2.1.234 plugin eval no longer swallows piped input
Both mention eval
v2.1.234 Plugin eval reference documents image grading and new exit codes
Both mention eval
v2.1.234 claude plugin eval handles termination signals and cleans up its output
Both mention eval

Plugin eval runs report exit codes, stderr and auth rejection

Related