Source Intelligence
Follow Discord
Sweep 28 Aug 2026 · 00:00Z Build v2.1.250 478 read Stable v2.1.236 Latest v2.1.250 Next v2.1.251 Feeds RSS JSON llms.txt
Reading a new release v2.1.251 Analysing changes · 2/5 Analysis pass · 22/27 agents ~1 min left 318 findings $9.88 so far

DisclaimerUnofficial, and not affiliated with Anthropic. Nearly all of this is read straight out of what ships: npm bundles, captured prompts, published docs. Anthropic's own notes go in verbatim, marked as theirs. The rest is my reading, and every entry carries the strings behind it. If one looks wrong, vote it down and say why.

Reference / CLI flags

--eval-dir

A Claude Code CLI flag, read out of the shipped bundle. It has been in the build since v2.1.235, including the newest one read.

What it is

Eval setup refuses directories with unverifiable ownership or symlinks, and cleans up after a failed setup.

Not Anthropic's. This is the line from the earliest changelog entry here that named it, v2.1.235.

Presence across releases

ReleasesReadingDeclared typeDefault
2.1.245 – 2.1.250 5 releases never mined — —
2.1.235 – 2.1.243 9 releases in the build — —

A row reading never mined is a release with no archived bundle behind it. Nothing was read out of it, so nothing here says whether the name was in it. A declared type or a default is only ever what that release's own bundle stated.

Changelog entries naming it

  1. v2.1.242claude plugin eval init refuses to scaffold in the wrong directory

    Eval scaffolding refuses to create files outside a plugin or skill folder unless you pass --eval-dir.

    named in this entry

  2. v2.1.235Eval setup refuses directories it cannot vouch for

    Eval setup refuses directories with unverifiable ownership or symlinks, and cleans up after a failed setup.

    named in this entry

  3. v2.1.235Eval warns when its directory overlaps the plugin's own component paths

    Eval warns when your eval directory overlaps paths the plugin already loads components from.

    named in this entry

  4. v2.1.235plugin eval can be pointed at a different case directory

    Point plugin eval at any case directory with --eval-dir, still behind an early-access gate or env var.

    named in this entry

  5. v2.1.235plugin eval now tells you when a nearby plugin is not being loaded

    Plugin eval now tells you when it deliberately skipped loading a nearby plugin it doesn't trust.

    named in this entry

  6. v2.1.235--eval-dir on the eval command, with warnings that name the fix

    The eval command reads its directory from a flag or the manifest, warning clearly when the value is malformed.

    named in this entry

  7. v2.1.235Eval runs record their output directory and show per-case advisories

    Eval runs record which directory they used and show per-case warning notes in the HTML report.

    named in this entry

Documentation pages

None of Anthropic's documentation pages this site has a copy of carries this name.

Names beside it

Every CLI flag in the inventory starting --eval, up to twelve of them.

--eval-mock-serverThis is version 2.1.242, and the CLI gained an --eval-mock-server mode taking two arguments. never mined

Read out of the published npm bundle release by release, and out of Anthropic's own documentation as this site captured it. Nothing on this page is a description anybody here wrote about what the CLI flag does. All CLI flags.