{"version":"2.1.251","anchor":"eval-failures-say-whether-they-can-be-reproduced","canonical_anchor":"eval-failures-say-whether-they-can-be-reproduced","heading":"Eval failures say whether they can be reproduced","tier":"internal","area":"Plugin Eval","scope":null,"heads_up":null,"url":"https:\/\/changelogs.core-directive.com\/v\/2.1.251\/e\/eval-failures-say-whether-they-can-be-reproduced","release_url":"https:\/\/changelogs.core-directive.com\/v\/2.1.251","markdown":"### Eval failures say whether they can be reproduced\n\nEval failures now say whether they're reproducible and flag runs killed before reporting.\n\n**What**\n\nWhen the eval harness's integrity check fails, each failure is now labelled as provokable or unprovokable, and a note is added when the child agent was killed before it could report which calls it refused, so a killed run is no longer indistinguishable from a genuine mismatch.\n\n**Evidence**\n\n`the child was killed before it reported which calls it refused itself`\n\n- Area: Plugin Eval\n- Tier: Under the hood\n- Useful: 1\/5\n- Signal: 2\/5"}