Stop 18 · Evaluation, Safety & Responsible AITopics · 2/9

Human Evaluation & LLM-as-Judge

In short

Is this answer helpful? Is the tone right? No word-count metric can say. Humans can, but they're slow. So teams use an LLM as a judge, and humans to check the judge.

Using human raters, or another LLM prompted to score outputs, to evaluate quality dimensions automated metrics can't capture — like helpfulness, tone and factual correctness.

The blob sits at a judges' table holding up a score card while a small robot judge beside it does the same.

Think of it like this

Judges at a debate competition

Debate judges don't count words. They score each speaker on a sheet: argument, evidence, delivery, out of 10 each. Two judges compare notes when they disagree. LLM-as-judge is a tireless judge with the same scoring sheet, and you still sit in to check its calls.

Watch it happen

Scoring one answer with a judge

Your app

Judge LLM

Human

Q + answer + rubric

1/5The app's answer is sent to a judge model along with the question and a written rubric.

The big ideas

What you need to know

Write a clear rubric

Define each score level with examples. 'Helpful' alone means different things to different raters.

Pairwise is easier

Asking 'which of these two is better?' is often more reliable than a 1 to 10 score.

Judges have biases

LLM judges can favour longer answers, the first option shown, or their own style. Swap order and check.

Calibrate with humans

Measure how often the judge agrees with people before you trust its numbers at scale.

See it in code

Walk through the code

text

1You are grading an answer from a college helpdesk bot.2 3Question: {question}4Reference facts: {facts}5Answer to grade: {answer}6 7Score each from 1 to 5:8- correct: 5 = every claim matches the facts, 1 = main claim is wrong9- complete: 5 = answers every part of the question10- tone: 5 = polite, clear and short11 12First write 2 to 3 sentences of reasoning.13Then output JSON only: {"correct": n, "complete": n, "tone": n}

1/3Give the judge the question, the facts it should check against, and the answer.

A judge prompt. Fill the slots for each test case, send it to a strong model and parse the JSON at the end.

Save yourself the pain

Common mistakes

Oops

Letting the same model grade its own answers and trusting it fully.

Do this instead

Models can favour their own outputs. Use a different strong model as judge, and check it against human scores.

Oops

Asking the judge 'Is this good? Yes or no.'

Do this instead

Use a rubric with defined levels and separate criteria, so you can see what improved.

Your mission

Judge the judge

~40 min

Quick check

Your LLM judge always gives higher scores to longer answers. What's this called?

Go deeper

1Design a simple rubric and manually evaluate a batch of model outputs against it.
2Try LLM-as-judge: prompt a strong model to score outputs against the same rubric.
3Learn LLM-as-judge's known biases (favoring longer or more confident-sounding answers).