develop-tests changedtest-and-evaluate/develop-tests
Recorded here
Lines+6added
Lines−6removed
From line
98
where the diff opens
First seen
14 Aug 2026
this site's first read of the page
Recorded edits11to this page, all time
The whole hunk
from line 98, old and new numbered
/
from line 98
9898Most use cases need multidimensional evaluation along several success criteria.
9999
100100<Accordion title="Example multidimensional criteria for sentiment analysis">
101 | | Criteria |
102 | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
103 | Bad | The model should classify sentiments well |
104 | Good | On a held-out test set of 10,000 diverse Twitter posts, the sentiment analysis model should achieve: - an F1 score of at least 0.85 - 99.5% of outputs are non-toxic - 90% of errors would cause inconvenience, not egregious error\* - 95% response time \< 200ms |
101 | | Criteria |
102 | ---- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
103 | Bad | The model should classify sentiments well |
104 | Good | On a held-out test set of 10,000 diverse Twitter posts, the sentiment analysis model should achieve:<br />- an F1 score of at least 0.85<br />- 99.5% of outputs are non-toxic<br />- 90% of errors would cause inconvenience, not egregious error\*<br />- 95% response time \< 200ms |
105105
106106 \*In reality, you would also define what "inconvenience" and "egregious" mean.
107107</Accordion>
from line 1549
15491549 );
15501550 $scoreText = trim(contentText($response));
15511551 if (filter_var($scoreText, FILTER_VALIDATE_INT) === false) {
1552 throw new RuntimeException("Unexpected rating from grader: {$scoreText}");
1552 throw new \RuntimeException("Unexpected rating from grader: {$scoreText}");
15531553 }
15541554 return (int) $scoreText;
15551555 }
from line 2719
27192719 );
27202720 $scoreText = trim(contentText($response));
27212721 if (filter_var($scoreText, FILTER_VALIDATE_INT) === false) {
2722 throw new RuntimeException("Unexpected rating from grader: {$scoreText}");
2722 throw new \RuntimeException("Unexpected rating from grader: {$scoreText}");
27232723 }
27242724 return (int) $scoreText;
27252725 }
No line in this hunk matches that.