{"version":"2.1.234","anchor":"plugin-eval-runs-are-isolated-from-project-settings-and-take","canonical_anchor":"plugin-eval-runs-are-isolated-from-project-settings-and-take","heading":"Plugin eval runs are isolated from project settings and take prompts over stdin","tier":"notice","area":"Plugin Evals","scope":null,"heads_up":null,"url":"https:\/\/changelogs.core-directive.com\/v\/2.1.234\/e\/plugin-eval-runs-are-isolated-from-project-settings-and-take","release_url":"https:\/\/changelogs.core-directive.com\/v\/2.1.234","markdown":"### Plugin eval runs are isolated from project settings and take prompts over stdin\n\nPlugin eval prompts now go over stdin and runs ignore project settings, so long prompts survive intact.\n\n**What**\n\nEach eval case's prompt is now written to the child process's standard input instead of being passed as a command-line argument, so long or shell-awkward prompts survive intact. The runner also passes `--setting-sources user`, so project and local settings cannot perturb a run.\n\n**Details**\n\n- The child is spawned with `stdio: [\"pipe\", \"pipe\", \"pipe\"]`; errors writing to stdin are ignored.\n- Model, allowed tools and appended system prompt are now passed in `=` form.\n- The `--ablation` help text was rewritten: the default is the with-without comparison whenever a plugin resolves by name or from the target path, and under that mode graders marked with-only signal that the plugin fired rather than counting toward the score.\n\n**Evidence**\n\n`stdio: [\"pipe\", \"pipe\", \"pipe\"]`, `Run a no-plugin baseline arm and report the score delta`\n\n- Area: Plugin Evals\n- Names: `--setting-sources user`\n- Tier: You'll notice\n- Useful: 3\/5\n- Signal: 2\/5"}