What
Running eval scaffolding outside a plugin or skill folder now errors and exits 1 instead of creating files where they do not belong, unless you pass --eval-dir. Output directories are also vetted: a symlinked base is refused, and when ownership vetting is on, so is a directory owned by another user.
Details
- The refusal is recorded as
cwd_not_a_plugin. - Path checks throw an escape error rather than silently writing outside the intended base.
Usage
claude plugin eval init --eval-dir ./evalsEvidence
is not a plugin or skill folder
Strings lifted out of the shipped bundle, so the claim above can be checked against them.
Names in the bundleclaude plugin eval init--eval-dir
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Confirmed since
Anthropic's documentation has since written up --eval-dir, on Plugins reference.
| `experimental.evals` | string\|array | Directory below the plugin root that holds the plugin's [eval cases](/docs/en/plugin-evals#use-a-different-eval-directory), when it isn't the default `evals/`. `claude plugin eval --eval-dir` overri…plugins-reference see the edit
Confirmed since
Anthropic's documentation has since written up claude plugin eval init, on Error reference.
You ran [`claude plugin eval`](/docs/en/plugin-evals) or `claude plugin eval init` and it exited 1 with one of these messages before doing anything:errors see the edit
One source agreesOne thing we can check says the same as this entry.
Anthropic's documentation agrees
Anthropic's documentation has since written up claude plugin eval init, on Error reference.