RAG-based Q&A Chatbot
Part of the AI Engineer Roadmap
In short
Build a chatbot that answers questions from real documents, like your college handbook, with citations. It's the most asked-for AI project in interviews because it combines every skill from this phase.
A chatbot that answers questions grounded in a specific document set — the flagship RAG project and one of the most in-demand AI engineer portfolio pieces.
Think of it like this
The admission helpdesk
During admissions, the helpdesk person doesn't remember every rule. When you ask about hostel fees, she flips to the right page in the binder and tells you, pointing at the line. Your chatbot should work exactly like that helpful person with the binder.
Watch it happen
Build plan
1/5Load PDFs, extract text page by page, and clean headers and footers.
The big ideas
What you need to know
Citations are a must
Show the source file and page for each answer. It builds trust and makes debugging easy.
Graceful 'I don't know'
Test questions the documents don't cover. A good bot says so instead of guessing.
Follow-up questions
Rewrite follow-ups like 'and for PG students?' into standalone questions before retrieving.
Show your numbers
Put recall@5 and a few example answers in the README. It shows you measured, not just demoed.
See it in code
Walk through the code
1def rewrite_followup(history: list[dict], question: str) -> str:2 if not history:3 return question4 recent = "\n".join(f"{m['role']}: {m['content']}" for m in history[-4:])5 return ask_llm(6 "Rewrite the last question so it makes sense on its own. Return only the question.\n"7 f"<chat>\n{recent}\n</chat>\nLast question: {question}"8 )9 10def chat(history, question):11 standalone = rewrite_followup(history, question)12 return answer(standalone) # the retrieve-and-generate function from RAG Architecture1/3The first question needs no rewriting.
The trickiest bit of a chat-style RAG bot: turning 'and for PG?' into a full question before searching.
Save yourself the pain
Common mistakes
Oops
Retrieving with the raw follow-up 'what about PG students?'.
Do this instead
That has no topic words. Rewrite it into a standalone question first, then retrieve.
Oops
Demoing only questions you know it gets right.
Do this instead
Include tricky and unanswerable questions in your demo. It shows you understand the limits.
Your mission
First milestone: cited answers
Quick check
A user asks 'What's the hostel fee?' then 'And for girls' hostel?'. Why rewrite the second question before retrieval?
Go deeper
- 1Pick a real document set (docs, PDFs) and build the full ingestion-to-chat pipeline.
- 2Add source citations to every answer so users can verify claims.
- 3Test with questions the documents don't cover and confirm the bot says so instead of hallucinating.
More projects at this stop
Document Search Engine
Stuck on this topic? Ask an Insider
Get 1:1 guidance from people who've walked this exact path — on the InsideEdge app.
