{"version":"2.1.234","anchor":"plugin-eval-harness-can-grade-images","canonical_anchor":"plugin-eval-harness-can-grade-images","heading":"Plugin eval harness can grade images","tier":"use","area":"Plugin Evals","scope":null,"heads_up":null,"url":"https:\/\/changelogs.core-directive.com\/v\/2.1.234\/e\/plugin-eval-harness-can-grade-images","release_url":"https:\/\/changelogs.core-directive.com\/v\/2.1.234","markdown":"### Plugin eval harness can grade images\n\nEval graders can now judge images, so rendered slides and charts get scored on appearance.\n\n**What**\n\nA grader of type `llm` pointed at a PNG, JPEG, GIF or WebP file now shows that file to the judging model as an image, so rendered slides, charts and screenshots can be graded on what they look like. A `regex` grader over an image always fails and says to use an `llm` grader instead; over other binary files it still matches ASCII sequences.\n\n**Details**\n\n- The HTML report now explains cases that ran only one arm and therefore have no baseline to compare against.\n- Non-image binaries are unchanged: regex matching against extractable ASCII.\n\n**Evidence**\n\n``an \\`llm\\` grader shows it to the judge *as an image*``\n\n- Area: Plugin Evals\n- Names: `llm`, `regex`\n- Tier: Use it now\n- Useful: 2\/5\n- Signal: 3\/5"}