Eval runs record which directory they used and show per-case warning notes in the HTML report.
What's wrong with this entry?
plugin eval and plugin eval init accept --eval-dir; the value is verified (failing with "--eval-dir cannot be verified: "), passed into the generated interview prompt and the child process that runs the cases, and stored as out_dir in the run metadata. The scored HTML report renders per-case advisories as warning notes.
- Both commands stay behind the
pluginEvalearly-access gate.
--eval-dir cannot be verified:
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
Related
Other releases about the same thing. Found by shared names or similar wording; neither means one caused the other.
-
v2.1.233
Plugin command help text comes from one table
Both mention eval init
-
v2.1.242
claude plugin eval initrefuses to scaffold in the wrong directoryBoth mention eval init
-
v2.1.246
Project onboarding checklist gone
Both mention init