You'll noticeTier: how much it should matter to you
2Useful: my rating, 1 to 5
1Signal: worth watching, 1 to 5
ElsewhereArea: what it touches
ImprovementsKind: in v2.1.234,
ImprovementsSection of the release
What
A regex grader pointed at something that resolves to an image now fails with a message naming the media type and what to do instead: use an LLM grader focused on that file, or, if you are asserting something is absent, write a text rendering of it first and match against that.
Details
The eval authoring documentation now tells authors to set allowed_tools on every case alongside timeout_seconds.
Evidence
is an image (
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
