You'll notice
Useful3 Signal2
Plugin eval prompts now go over stdin and runs ignore project settings, so long prompts survive intact.
What's wrong with this entry?
Anonymous. No account, no email.
--setting-sources user
What
Each eval case's prompt is now written to the child process's standard input instead of being passed as a command-line argument, so long or shell-awkward prompts survive intact. The runner also passes --setting-sources user, so project and local settings cannot perturb a run.
Details
- The child is spawned with
stdio: ["pipe", "pipe", "pipe"]; errors writing to stdin are ignored. - Model, allowed tools and appended system prompt are now passed in
=form. - The
--ablationhelp text was rewritten: the default is the with-without comparison whenever a plugin resolves by name or from the target path, and under that mode graders marked with-only signal that the plugin fired rather than counting toward the score.
Evidence
stdio: ["pipe", "pipe", "pipe"], Run a no-plugin baseline arm and report the score delta
Strings lifted out of the shipped bundle, so the claim above can be checked against them.