Use case
Grading model output
Score another model's answer against a rubric you wrote, and keep the number.
What matters
The grader should return a level and a confidence, not a pep talk. The rubric has to be yours.
accuracy · structured-output
Top picks
- #1TypeSafe: Jev 1.14
Score questions are the product: an ordered rubric comes back as probabilities and an expected score.
- #2Perplexity: Decider V1.2
Same score shape, and it can grade outputs that include images.
Budget pick
Perplexity: Decider V1.2
Open-weight pick
Perplexity: Decider V1.2
Pitfalls
- Asking a chat model 'rate this 1-5' and parsing the first digit is how rubrics drift.
- A high score is not a fact check. It is agreement with the rubric you sent.
Sample prompt
State: the question, the model's answer, and the source note. score 'grounded' as unsupported, partial, or grounded. Send 'partial' to a person.