Plugin eval checks your credentials up front and fails fast instead of dying partway through a run.
What's wrong with this entry?
The plugin eval CLI checks it can actually authenticate before running any cases, so a run fails fast with an explanation instead of partway through. It engages only for first-party auth in env-quad mode with none of ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN or ANTHROPIC_UNIX_SOCKET set.
- A failed setup or token exchange aborts the run with a message.
- Failures that look transient (HTTP 5xx, 429, 408, or DNS and socket errors) are downgraded to a warning and the run continues.
- 401 or 403 adds advice to check the federation rule and set
ANTHROPIC_WORKSPACE_ID.
workload identity federation could not obtain an access token
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
Related
Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.
-
v2.1.223
Settings-supplied API and credential env vars are stripped when ANTHROPIC_UNIX_SOCKET is set
Both mention auth token unix socket
-
v2.1.242
GitHub CLI login is detected with
gh auth tokenBoth mention auth token
-
v2.1.223
Policy-settings environment stripping is case-insensitive
Both mention unix socket