Stop 18 · Evaluation, Safety & Responsible AITopics · 6/9

Hallucination Detection & Mitigation

In short

LLMs sometimes state wrong facts in the same confident tone as right ones. You can't train this away fully, so you design the system to ground answers and catch the made-up bits.

Techniques for catching and reducing a model confidently stating false information — including grounding in retrieved sources, self-consistency checks, and citation requirements.

The blob raises an eyebrow and holds a magnifying glass up to a confident uncle gesturing grandly, marigolds at his feet.

Think of it like this

The uncle who knows everything

Every family wedding has that uncle with a confident answer to every question, from cricket stats to GDP. Some answers are right, some he made up, and he sounds identical either way. LLMs can be that uncle. Your job is to ask him for sources.

Watch it happen

Answering with a grounding check

Student

App

Rule book

LLM

Library hours on Sunday?

1/6A student asks the college helpdesk bot a factual question.

The big ideas

What you need to know

Ground in sources

Give the model the facts with RAG and tell it to answer only from them. This cuts most made-up facts.

Allow 'I don't know'

Explicitly tell the model it may say it doesn't know. Otherwise it tends to guess.

Require quotes and check them

Ask for the exact supporting quote, then check in code that it exists in the source.

Self-consistency

Ask the same question a few times. If answers disagree, confidence should be low.

See it in code

Walk through the code

python

1import json2 3context = ("The library is open 8 am to 10 pm on weekdays. "4           "It closes at 6 pm on Sundays.")5 6model_reply = '{"answer": "It closes at 6 pm on Sundays.", "quote": "It closes at 6 pm on Sundays."}'7reply = json.loads(model_reply)8 9if reply["quote"] and reply["quote"] in context:10    print("Grounded:", reply["answer"])11else:12    print("Quote not found in sources. Treat as unverified.")

1/3The retrieved source text the model was told to use.

A cheap grounding check. The model must return a quote, and your code confirms the quote is really in the retrieved text.

Save yourself the pain

Common mistakes

Oops

Thinking a bigger model means no hallucinations.

Do this instead

Bigger models hallucinate less on some tasks, but still do. Ground answers and check them.

Oops

Showing citations without checking they point to real text.

Do this instead

Models can invent citations too. Verify each one against the actual source.

Your mission

Catch a hallucination

~30 min

Quick check

Which change most reduces hallucinated facts in a helpdesk bot?

Go deeper

1Build a simple fact-check step that cross-references model claims against retrieved sources.
2Compare hallucination rates with and without RAG grounding on the same questions.
3Learn why no current technique eliminates hallucination completely — only reduces its rate.