{"version":"2.1.236","anchor":"plugin-eval-cases-must-have-a-prompt-and-unfilled-templates-","canonical_anchor":"plugin-eval-cases-must-have-a-prompt-and-unfilled-templates-","heading":"Plugin eval cases must have a prompt, and unfilled templates are rejected","tier":"use","area":"Plugin Evals","scope":null,"heads_up":null,"url":"https:\/\/changelogs.core-directive.com\/v\/2.1.236\/e\/plugin-eval-cases-must-have-a-prompt-and-unfilled-templates-","release_url":"https:\/\/changelogs.core-directive.com\/v\/2.1.236","markdown":"### Plugin eval cases must have a prompt, and unfilled templates are rejected\n\nPlugin eval cases now must include a prompt, and untouched init placeholders are rejected.\n\n**What**\n\nA plugin eval case that supplies only a session history file is now refused; you need `execution.prompt` in `case.yaml` or a `prompt.md` body. A case or grader still containing the placeholder line written by `init` is refused with an error explaining that.\n\n**Details**\n\n- Directory scanning for cases was rewritten, and paths written inside a case are now resolved with a check that they do not escape the case folder.\n\n**Evidence**\n\n`execution.prompt is required (a prompt.md body, or execution.prompt in case.yaml)`\n\n- Area: Plugin Evals\n- Names: `execution.prompt`, `case.yaml`, `prompt.md`\n- Tier: Use it now\n- Useful: 3\/5\n- Signal: 2\/5"}