Plugin eval checks your credentials first and aborts instead of failing every case.
An auth preflight for claude plugin eval, behind the early-access gate.
tengu_walnut_spire Off in both readingsThe flag server returned off for the account this site reads and for the anonymous baseline. A reading of off cannot rule out a rollout these two readings sit outside of.
This account: off · anonymous baseline: off · compiled default in v2.1.233: on
These values were read against a different version of Claude Code, so treat them as the nearest reading available instead of one taken on this release.
Read once, for one account on one subscription tier, against v2.1.233. It isn't a statement about your account. What a flag value here can and cannot tell you
What's wrong with this entry?
claude plugin eval now runs an authentication preflight up front. If it fails, the whole suite aborts with exit code 2 and a report status of auth_failed rather than burning every case on a broken credential. A preflight problem that is not fatal is printed as a notice and the run continues.
- The result object carries an
authPreflightFailedflag alongside the report status. - Plugin eval as a whole is early access, gated on
tengu_walnut_spireorCLAUDE_CODE_WALNUT_SPIRE.
authentication check failed before running any case
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
Related
Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.
-
v2.1.234
plugin evalno longer swallows piped inputBoth mention eval
-
v2.1.234
Plugin eval reference documents image grading and new exit codes
Both mention eval
-
v2.1.234
claude plugin evalhandles termination signals and cleans up its outputBoth mention eval