plugin eval warns when writing results inside a plugin's own directory could get picked up as plugin components
TierYou'll noticehow much it should matter to you
Useful2my rating, 1 to 5
Signal1worth watching, 1 to 5
AreaPlugin Evalwhat it touches
KindImprovementsin v2.1.235,
You'll notice
plugin eval warns when writing results inside a plugin's own directory could get picked up as plugin components
When eval results are written under the plugin itself (no --output-dir given), the command now checks the destination against the plugin's layout and warns that files written there may be loaded as plugin components. Case discovery also refuses to follow symlinked directories, and logs when a directory has no prompt.md or case.yaml, so it's scanned for subcases rather than mistakenly treated as one.
Since it was published
The entry above is what we published on the day. These lines were added later, as Anthropic's own pages caught up, and they sit beside the original rather than replacing it.
Confirmed sinceAnthropic's documentation has since written up --output-dir, on Plugins reference.This table lists the options most runs use. Run `claude plugin eval --help` for the complete set, including `--case`, `--tag`, `--output-dir`, `--report`, `--allow-real-servers`, `--keep-temp`, and `--verbose`.plugins-referencesee the edit
How sure we are
One source agreesOne thing we can check says the same as this entry.
Anthropic's documentation agreesAnthropic's documentation has since written up --output-dir, on Plugins reference.