What
A grader of type llm pointed at a PNG, JPEG, GIF or WebP file now shows that file to the judging model as an image, so rendered slides, charts and screenshots can be graded on what they look like. A regex grader over an image always fails and says to use an llm grader instead; over other binary files it still matches ASCII sequences.
Details
- The HTML report now explains cases that ran only one arm and therefore have no baseline to compare against.
- Non-image binaries are unchanged: regex matching against extractable ASCII.
Evidence
`an \llm\ grader shows it to the judge as an image`
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Added since
A small documentation edit on Connect Claude Code to an LLM gateway touched a line naming regex after this was published.
| `400` errors on every request, in the gateway's own words rejecting a tool's input schema or its `pattern`, on Claude Code v2.1.265 through v2.1.267 | In a gradual rollout on those versions, the [Artifact tool](/docs/en/artifacts#availab…llm-gateway-connect see the edit
Added since
A small documentation edit on How Claude Code uses prompt caching touched a line naming llm after this was published.
Claude Code also appends system context mid-conversation, such as file-change notices, and marks that block for caching on every provider and connection unless you set [`CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS`](/docs/en/llm-gateway-protoco…prompt-caching see the edit
Confirmed since
Anthropic's documentation has since written up regex, on Batches.
- `type: "tool_search_tool_regex_20251119" or "tool_search_tool_regex"`api/messages/batches see the edit
Confirmed since
Anthropic's documentation has since written up llm, on User Profiles.
- `type: "enrollment_url"`api/beta/user_profiles see the edit
One source agreesOne thing we can check says the same as this entry.
Anthropic's documentation agrees
Anthropic's documentation has since written up llm, on User Profiles.