{"version":"2.1.242","anchor":"plugin-eval-docs-scores-from-untrusted-suites-are-advisory","canonical_anchor":"plugin-eval-docs-scores-from-untrusted-suites-are-advisory","heading":"Plugin eval docs: scores from untrusted suites are advisory","tier":"notice","area":"Elsewhere","scope":null,"heads_up":null,"url":"https:\/\/changelogs.core-directive.com\/v\/2.1.242\/e\/plugin-eval-docs-scores-from-untrusted-suites-are-advisory","release_url":"https:\/\/changelogs.core-directive.com\/v\/2.1.242","markdown":"### Plugin eval docs: scores from untrusted suites are advisory\n\nEval docs now warn that scores from untrusted suites granted Bash should be treated as advisory.\n\n**What**\n\nThe eval sandbox documentation now warns that the harness judges a run from the child process's output and files, which a shell-granted process running as you can also reach. Scores from an untrusted suite run with `--allow-tools Bash`, or any other grant that can execute code, should be treated as advisory unless the run had operating-system-level isolation.\n\n**Details**\n\n- Text only; the sandbox itself behaves as before.\n\n**Evidence**\n\n`treat scores from an untrusted suite run`\n\n- Area: Elsewhere\n- Names: `--allow-tools Bash`\n- Tier: You'll notice\n- Useful: 2\/5\n- Signal: 2\/5"}