What
claude plugin eval runs a plugin against a set of test cases and scores the results. Before a run it now checks git --version and stops with a message that git is too old if the version is below 2.31. Older git ignores GIT_CONFIG_COUNT, a way of passing git settings through the environment, and eval relies on it to switch off the repository's hooks and helper programs.
Eval also sets git so that it prints output without a pager, never opens an editor, and uses no credential helper (core.pager=cat, core.editor=true and an empty credential.helper).
Why
This keeps a repository's own scripts from running during an eval, but eval can now fail on machines with an older git that it used to run on.
Names in the bundleclaude plugin evalGIT_CONFIG_COUNT
claude plugin eval
Test plugins with evals modified, high confidence
The `claude plugin eval` shell command runs your [plugin](/docs/en/plugins/overview) against a suite of test cases and scores the results. Each case is a realistic prompt plus one or more graders. A grader is a pass/fail check on what Clau…see the edit
claude plugin eval
Extend Claude with skills modified, high confidence
Two tools automate that comparison. For a skill that ships in a [plugin](/docs/en/plugins/overview), [`claude plugin eval`](/docs/en/plugin-evals) runs each prompt in an isolated session with and without the plugin, scores it with graders …see the edit
claude plugin eval
Create custom subagents modified, high confidence
If the subagent ships in a [plugin](/docs/en/plugins/overview), you can measure how reliably Claude delegates to it across realistic prompts instead of checking one at a time: [`claude plugin eval`](/docs/en/plugin-evals) runs each prompt …see the edit
claude plugin eval
Create custom skills modified, high confidence
If the skill is part of a plugin, you can also test the whole plugin from the Claude Code command line with [`claude plugin eval`](https://code.claude.com/docs/en/plugin-evals), which grades eval cases you write and compares against a run …see the edit
Two sources agreeTwo things we can check say the same as this entry.
Anthropic's documentation agrees
claude plugin eval on Test plugins with evals
Anthropic's release notes agree
Fixed installed_plugins.json being rewritten, losing records, when it holds a record this version cannot read; claude plugin commands now…