Stop 12 · Prompt EngineeringTopics · 9/10

Handling Hallucinations via Prompting

In short

LLMs sometimes state wrong facts with full confidence, even inventing citations. You can't remove this completely, but good prompting cuts it a lot: give sources, allow 'I don't know', and ask for evidence.

Prompt-level techniques (asking for citations, allowing 'I don't know', grounding in provided context) that reduce — though never fully eliminate — a model inventing false information.

The blob in a viva points to a page in an open reference book while an examiner listens.

Think of it like this

The honest viva

In a viva, a nervous student might bluff an answer. A smart one says, 'Sir, it's on page 42 here,' and points to the textbook, or admits, 'I'm not sure.' Examiners trust the second student more. Prompt your model to behave like that student.

Watch it happen

A grounded-answer recipe

Sources

Quote

Answer

Or unsure

Verify

1/5Put the relevant documents in the prompt, and tell the model to use only them.

The big ideas

What you need to know

Ground it in sources

Models make up less when the answer is in the prompt. That's why RAG exists.

Permission to not know

Explicitly allow 'I don't know'. Without it, the model may feel it must always answer.

Quotes before claims

Asking for supporting quotes first makes unsupported claims easier to spot, and you can check quotes with code.

Lower temperature for facts

Low temperature reduces random wandering. It doesn't fix wrong knowledge, but helps consistency.

See it in code

Walk through the code

text

1Answer the question using ONLY the documents below.2 3Steps:41. Copy the exact sentences that answer the question into <quotes>.52. Answer in 2 lines inside <answer>, citing [doc number].63. If no sentence answers it, write <answer>Not found in the documents.</answer>7 8<doc 1>Library hours are 8 am to 10 pm on weekdays.</doc 1>9<doc 2>During exams, the reading room stays open until midnight.</doc 2>10 11Question: Is the library open on Sundays?

1/3Restrict the model to the provided text.

The documents don't mention Sundays, so a well-behaved model should reply 'Not found' instead of guessing.

Save yourself the pain

Common mistakes

Oops

Asking 'Give me 5 research papers on X with links' and trusting the list.

Do this instead

Models can invent papers and URLs. Search a real database, then let the model summarise what you found.

Oops

Thinking 'Don't hallucinate' in the prompt solves it.

Do this instead

Give sources, allow 'not found', ask for quotes and verify. Instructions alone don't change what it knows.

Your mission

Catch a made-up answer

~20 min

Quick check

Which prompt change most directly reduces made-up answers in a document Q&A bot?

Go deeper

1Test prompts that explicitly permit 'I don't know' and observe the effect on made-up answers.
2Practice grounding prompts in provided context and instructing the model not to go beyond it.
3Learn why prompting alone can't fully solve hallucination — this motivates RAG in a later phase.