Sweep 19 Sep 2026 · 02:36Z Build v2.1.278 500 read Stable v2.1.267 Latest v2.1.278 Next v2.1.278 Feeds RSS JSON llms.txt Unofficial
Claude Code v2.1.269 ·

Eval runner concurrency now coordinated through a token gate

Eval workers now share a token gate that serializes login/credential refresh and was threaded into a simplified single-run eval loop

TierUnder the hoodhow much it should matter to you
Useful2my rating, 1 to 5
Signal2worth watching, 1 to 5
AreaPlugin Evalwhat it touches
KindInternal Changesin v2.1.269,
Group of 2 Under the hood

Eval workers now share a token gate that serializes login/credential refresh and was threaded into a simplified single-run eval loop

What

  • The eval case-runner (used to run evaluation test cases) was simplified from a loop that repeated a case N times with per-iteration cost and abort checks into a single-run function, and it now releases a tokenGate when one was acquired, only short-circuiting on an auth failure if no other reason for stopping has already been recorded.
  • A new token rotation gate (a mutex-like mechanism with enter/leave/rotate actions) was added and is used by the eval login and gateway token refresh path, so that when multiple eval workers share the same credentials, only one of them refreshes the token at a time. It includes retry backoff and a "skip until" cache so a failing refresh isn't retried too aggressively.

Why

This prevents concurrent eval workers from stepping on each other while refreshing shared login credentials, and cleans up how the eval runner tracks whether it's still allowed to keep going.

How sure we are
Something disagreesSomething we can check disagrees with this entry, or the writer said they could not settle it.
The writer flagged doubtThe finding does not say what change in behavior, if any, this produces for someone running `claude plugin eval`.

See this entry in the whole of v2.1.269 →