A failed login stops the whole eval suite instead of wasting every case.
Auth-failure abort and report banner for plugin eval, behind the early-access check.
What's wrong with this entry?
An eval run that cannot authenticate, or whose credential is rejected before any case completes, now aborts the whole suite instead of burning every remaining case on the same broken credential. The HTML report banner explains the stop, the CLI prints "stopped: authentication failed", and the outcome is reported separately from the two existing partial reasons, a cost ceiling and interruption. plugin eval remains behind an early-access check.
- Individual case results carry an auth-rejected outcome with its own label in the run-status table.
- The suite aborts once the first run comes back auth-rejected.
- A new environment variable marks the child process spawned for the
plugin eval initinterview, and its presence suppresses report publishing. - The command's long description moved into the shared command-help table.
A run could not authenticate (or the credential was rejected before any run), so the suite stopped before every case finished., a run could not authenticate \u2014 every remaining run would fail the same way, so the suite stops here. Fix the credential above and re-run., stopped: authentication failed
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
Related
Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.
-
v2.1.234
plugin evalno longer swallows piped inputBoth mention eval
-
v2.1.234
Plugin eval reference documents image grading and new exit codes
Both mention eval
-
v2.1.234
claude plugin evalhandles termination signals and cleans up its outputBoth mention eval